Max level shown:
12025-04-30T06:43:31.129ZDEBGtest_unauthorized (init_with_steps): Running step start_crdb
22025-04-30T06:43:31.130ZDEBGtest_unauthorized: Starting CRDB
32025-04-30T06:43:31.160ZINFOtest_unauthorized: cockroach temporary directory: /var/tmp/omicron_tmp/.tmpdIrcPv
42025-04-30T06:43:31.160ZINFOtest_unauthorized: cockroach: copying from seed tarball (/var/tmp/omicron_tmp/crdb-base-build/f234518892194a05ad8767dc97f01a61a55baf1431942a314ce8b0f72b18ec2e.tar) to storage directory (/var/tmp/omicron_tmp/.tmpdIrcPv/data)
52025-04-30T06:43:31.178ZINFOtest_unauthorized: cockroach command line: cockroach start-single-node --insecure --http-addr=:0 --store=path=/var/tmp/omicron_tmp/.tmpdIrcPv/data,ballast-size=0 --listen-addr [::1]:0 --listening-url-file /var/tmp/omicron_tmp/.tmpdIrcPv/listen-url
62025-04-30T06:43:31.178ZINFOtest_unauthorized: cockroach environment: BUILDOMAT_JOB_ID=01JT2JWKGHF3JHFMW0JR13NSA6 BUILDOMAT_TASK_ID=4 CARGO=/home/build/.rustup/toolchains/1.86.0-x86_64-unknown-illumos/bin/cargo CARGO_HOME=/home/build/.cargo CARGO_INCREMENTAL=0 CARGO_MANIFEST_DIR=/work/oxidecomputer/omicron/nexus CARGO_PKG_AUTHORS= CARGO_PKG_DESCRIPTION= CARGO_PKG_HOMEPAGE= CARGO_PKG_LICENSE=MPL-2.0 CARGO_PKG_LICENSE_FILE= CARGO_PKG_NAME=omicron-nexus CARGO_PKG_REPOSITORY= CARGO_PKG_RUST_VERSION= CARGO_PKG_VERSION=0.1.0 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=1 CARGO_PKG_VERSION_PATCH=0 CARGO_PKG_VERSION_PRE= CARGO_TERM_COLOR=always CI=true CRDB_SEED_TAR=/var/tmp/omicron_tmp/crdb-base-build/f234518892194a05ad8767dc97f01a61a55baf1431942a314ce8b0f72b18ec2e.tar GITHUB_REPOSITORY=oxidecomputer/omicron GITHUB_SHA=531912b3fc900900bc7bd18f5eac1ce1e22cb1cd GOTRACEBACK=crash HOME=/home/build LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8 LD_LIBRARY_PATH=/work/oxidecomputer/omicron/target/debug/build/aws-lc-sys-6532f52db341a7cc/out/build/artifacts:/work/oxidecomputer/omicron/target/debug/build/blake3-403ee89fc24db8f8/out:/work/oxidecomputer/omicron/target/debug/build/bzip2-sys-aece66bfb5c8df92/out/lib:/work/oxidecomputer/omicron/target/debug/build/libgit2-sys-5fe90ecb72e1690f/out/build:/work/oxidecomputer/omicron/target/debug/build/ring-fb368fe91b2bd145/out:/work/oxidecomputer/omicron/target/debug/build/tofino-ffe28f7f2301d0c2/out:/work/oxidecomputer/omicron/target/debug/build/zstd-sys-a7fb19a1e17b70f6/out:/work/oxidecomputer/omicron/target/debug/deps:/work/oxidecomputer/omicron/target/debug:/home/build/.rustup/toolchains/1.86.0-x86_64-unknown-illumos/lib/rustlib/x86_64-unknown-illumos/lib:/home/build/.rustup/toolchains/1.86.0-x86_64-unknown-illumos/lib LOGNAME=build NEXTEST=1 NEXTEST_BIN_EXE_nexus=/work/oxidecomputer/omicron/target/debug/nexus NEXTEST_BIN_EXE_schema-updater=/work/oxidecomputer/omicron/target/debug/schema-updater NEXTEST_EXECUTION_MODE=process-per-test NEXTEST_LD_LIBRARY_PATH=/work/oxidecomputer/omicron/target/debug/build/aws-lc-sys-6532f52db341a7cc/out/build/artifacts:/work/oxidecomputer/omicron/target/debug/build/blake3-403ee89fc24db8f8/out:/work/oxidecomputer/omicron/target/debug/build/bzip2-sys-aece66bfb5c8df92/out/lib:/work/oxidecomputer/omicron/target/debug/build/libgit2-sys-5fe90ecb72e1690f/out/build:/work/oxidecomputer/omicron/target/debug/build/ring-fb368fe91b2bd145/out:/work/oxidecomputer/omicron/target/debug/build/tofino-ffe28f7f2301d0c2/out:/work/oxidecomputer/omicron/target/debug/build/zstd-sys-a7fb19a1e17b70f6/out:/work/oxidecomputer/omicron/target/debug/deps:/work/oxidecomputer/omicron/target/debug:/home/build/.rustup/toolchains/1.86.0-x86_64-unknown-illumos/lib/rustlib/x86_64-unknown-illumos/lib:/home/build/.rustup/toolchains/1.86.0-x86_64-unknown-illumos/lib NEXTEST_PROFILE=ci NEXTEST_RUN_ID=29735cc2-0883-4059-8ca2-1da53f01ede3 NEXTEST_TEST_GLOBAL_SLOT=7 NEXTEST_TEST_GROUP=@global NEXTEST_TEST_GROUP_SLOT=none OUT_DIR=/work/oxidecomputer/omicron/target/debug/build/omicron-nexus-553062f859dbbfa0/out PATH=/work/oxidecomputer/omicron/out/mgd/root/opt/oxide/mgd/bin:/work/oxidecomputer/omicron/out/dendrite-stub/bin:/work/oxidecomputer/omicron/out/clickhouse:/work/oxidecomputer/omicron/out/cockroachdb/bin:/home/build/.cargo/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/ooce/bin:/opt/ooce/sbin PWD=/work/oxidecomputer/omicron RUSTC_BOOTSTRAP=1 RUSTDOCFLAGS=--document-private-items -D warnings RUSTFLAGS=-D warnings -C link-arg=-R/usr/platform/oxide/lib/amd64 RUSTUP_HOME=/home/build/.rustup RUSTUP_TOOLCHAIN=1.86.0-x86_64-unknown-illumos RUST_BACKTRACE=1 RUST_RECURSION_COUNT=1 SHLVL=1 SSL_CERT_DIR=/usr/ssl/certs SSL_CERT_FILE=/etc/ssl/cacert.pem TMPDIR=/var/tmp/omicron_tmp TZ=UTC USER=build _=/usr/bin/ptime __NEXTEST_ATTEMPT=1
72025-04-30T06:43:32.693ZINFOtest_unauthorized: cockroach pid: 27219
82025-04-30T06:43:32.693ZINFOtest_unauthorized: cockroach listen URL: postgresql://root@[::1]:46222/omicron?sslmode=disable
92025-04-30T06:43:32.694ZDEBGtest_unauthorized (init_with_steps): Running step start_clickhouse
102025-04-30T06:43:32.694ZDEBGtest_unauthorized: Starting Clickhouse
112025-04-30T06:43:33.421ZDEBGtest_unauthorized (init_with_steps): Running step start_gateway_switch0
122025-04-30T06:43:33.421ZDEBGtest_unauthorized: Starting Management Gateway
132025-04-30T06:43:33.560ZDEBGtest_unauthorized (init_with_steps): Running step start_dendrite_switch0
142025-04-30T06:43:33.560ZDEBGtest_unauthorized: Starting Dendrite for switch0
152025-04-30T06:43:33.612ZDEBGtest_unauthorized (init_with_steps): Running step start_mgd_switch0
162025-04-30T06:43:33.612ZDEBGtest_unauthorized: Starting mgd for switch0
172025-04-30T06:43:33.718ZDEBGtest_unauthorized: mgd port is 33198
182025-04-30T06:43:33.718ZDEBGtest_unauthorized (init_with_steps): Running step record_switch_dns
192025-04-30T06:43:33.718ZDEBGtest_unauthorized: Recording DNS for the switch zones sled_id = b6d65341-167c-41df-9b5c-41cded99c229 switch_location = switch0
202025-04-30T06:43:33.718ZDEBGtest_unauthorized (init_with_steps): Running step start_internal_dns
212025-04-30T06:43:33.718ZINFOtest_unauthorized (store): opening sled database path = /var/tmp/omicron_tmp/.tmplc3gTR
222025-04-30T06:43:33.841ZINFOtest_unauthorized (store): pruning trees for generations newer than 0
232025-04-30T06:43:33.842ZINFOtest_unauthorized (store): pruning trees for generations older than 0 keep = 3
242025-04-30T06:43:33.842ZINFOtest_unauthorized (dns): DNS server bound to address kind = dns local_address = [::1]:38709
252025-04-30T06:43:33.844ZDEBGtest_unauthorized (http): registered endpoint kind = dns local_addr = [::1]:39462 method = GET path = /config versions = all
262025-04-30T06:43:33.844ZDEBGtest_unauthorized (http): registered endpoint kind = dns local_addr = [::1]:39462 method = PUT path = /config versions = all
272025-04-30T06:43:33.844ZINFOtest_unauthorized (http): listening kind = dns local_addr = [::1]:39462
282025-04-30T06:43:33.845ZDEBGtest_unauthorized (http): successfully registered DTrace USDT probes kind = dns local_addr = [::1]:39462
292025-04-30T06:43:33.845ZDEBGtest_unauthorized (init_with_steps): Running step start_external_dns
302025-04-30T06:43:33.845ZINFOtest_unauthorized (store): opening sled database path = /var/tmp/omicron_tmp/.tmppsWrMl
312025-04-30T06:43:33.860ZINFOtest_unauthorized (store): pruning trees for generations newer than 0
322025-04-30T06:43:33.861ZINFOtest_unauthorized (store): pruning trees for generations older than 0 keep = 3
332025-04-30T06:43:33.861ZINFOtest_unauthorized (dns): DNS server bound to address kind = dns local_address = [::1]:51235
342025-04-30T06:43:33.861ZDEBGtest_unauthorized (http): registered endpoint kind = dns local_addr = [::1]:61288 method = GET path = /config versions = all
352025-04-30T06:43:33.861ZDEBGtest_unauthorized (http): registered endpoint kind = dns local_addr = [::1]:61288 method = PUT path = /config versions = all
362025-04-30T06:43:33.861ZINFOtest_unauthorized (http): listening kind = dns local_addr = [::1]:61288
372025-04-30T06:43:33.862ZDEBGtest_unauthorized (http): successfully registered DTrace USDT probes kind = dns local_addr = [::1]:61288
382025-04-30T06:43:33.862ZDEBGtest_unauthorized (init_with_steps): Running step start_nexus_internal
392025-04-30T06:43:33.863ZDEBGtest_unauthorized: Starting Nexus (internal API)
402025-04-30T06:43:33.863ZINFOtest_unauthorized: setting up nexus server
412025-04-30T06:43:33.866ZINFOtest_unauthorized (ServerContext): registering Oso class class = Action
422025-04-30T06:43:33.866ZINFOtest_unauthorized (ServerContext): registering Oso class class = AnyActor
432025-04-30T06:43:33.866ZINFOtest_unauthorized (ServerContext): registering Oso class class = AuthenticatedActor
442025-04-30T06:43:33.867ZINFOtest_unauthorized (ServerContext): registering Oso class class = BlueprintConfig
452025-04-30T06:43:33.867ZINFOtest_unauthorized (ServerContext): registering Oso class class = Database
462025-04-30T06:43:33.867ZINFOtest_unauthorized (ServerContext): registering Oso class class = DnsConfig
472025-04-30T06:43:33.867ZINFOtest_unauthorized (ServerContext): registering Oso class class = Fleet
482025-04-30T06:43:33.867ZINFOtest_unauthorized (ServerContext): registering Oso class class = Inventory
492025-04-30T06:43:33.867ZINFOtest_unauthorized (ServerContext): registering Oso class class = IpPoolList
502025-04-30T06:43:33.867ZINFOtest_unauthorized (ServerContext): registering Oso class class = ConsoleSessionList
512025-04-30T06:43:33.867ZINFOtest_unauthorized (ServerContext): registering Oso class class = DeviceAuthRequestList
522025-04-30T06:43:33.867ZINFOtest_unauthorized (ServerContext): registering Oso class class = SiloCertificateList
532025-04-30T06:43:33.867ZINFOtest_unauthorized (ServerContext): registering Oso class class = SiloIdentityProviderList
542025-04-30T06:43:33.867ZINFOtest_unauthorized (ServerContext): registering Oso class class = SiloUserList
552025-04-30T06:43:33.867ZINFOtest_unauthorized (ServerContext): registering Oso class class = TargetReleaseConfig
562025-04-30T06:43:33.867ZINFOtest_unauthorized (ServerContext): registering Oso class class = WebhookEventClassList
572025-04-30T06:43:33.867ZINFOtest_unauthorized (ServerContext): registering Oso class class = Project
582025-04-30T06:43:33.867ZINFOtest_unauthorized (ServerContext): registering Oso class class = Disk
592025-04-30T06:43:33.867ZINFOtest_unauthorized (ServerContext): registering Oso class class = Snapshot
602025-04-30T06:43:33.867ZINFOtest_unauthorized (ServerContext): registering Oso class class = ProjectImage
612025-04-30T06:43:33.867ZINFOtest_unauthorized (ServerContext): registering Oso class class = AffinityGroup
622025-04-30T06:43:33.867ZINFOtest_unauthorized (ServerContext): registering Oso class class = AntiAffinityGroup
632025-04-30T06:43:33.867ZINFOtest_unauthorized (ServerContext): registering Oso class class = Instance
642025-04-30T06:43:33.867ZINFOtest_unauthorized (ServerContext): registering Oso class class = IpPool
652025-04-30T06:43:33.868ZINFOtest_unauthorized (ServerContext): registering Oso class class = InstanceNetworkInterface
662025-04-30T06:43:33.868ZINFOtest_unauthorized (ServerContext): registering Oso class class = Vpc
672025-04-30T06:43:33.868ZINFOtest_unauthorized (ServerContext): registering Oso class class = VpcRouter
682025-04-30T06:43:33.868ZINFOtest_unauthorized (ServerContext): registering Oso class class = InternetGateway
692025-04-30T06:43:33.868ZINFOtest_unauthorized (ServerContext): registering Oso class class = InternetGatewayIpPool
702025-04-30T06:43:33.868ZINFOtest_unauthorized (ServerContext): registering Oso class class = InternetGatewayIpAddress
712025-04-30T06:43:33.868ZINFOtest_unauthorized (ServerContext): registering Oso class class = RouterRoute
722025-04-30T06:43:33.868ZINFOtest_unauthorized (ServerContext): registering Oso class class = VpcSubnet
732025-04-30T06:43:33.868ZINFOtest_unauthorized (ServerContext): registering Oso class class = FloatingIp
742025-04-30T06:43:33.868ZINFOtest_unauthorized (ServerContext): registering Oso class class = Image
752025-04-30T06:43:33.868ZINFOtest_unauthorized (ServerContext): registering Oso class class = SiloImage
762025-04-30T06:43:33.868ZINFOtest_unauthorized (ServerContext): registering Oso class class = AddressLot
772025-04-30T06:43:33.868ZINFOtest_unauthorized (ServerContext): registering Oso class class = Blueprint
782025-04-30T06:43:33.868ZINFOtest_unauthorized (ServerContext): registering Oso class class = LoopbackAddress
792025-04-30T06:43:33.868ZINFOtest_unauthorized (ServerContext): registering Oso class class = Certificate
802025-04-30T06:43:33.868ZINFOtest_unauthorized (ServerContext): registering Oso class class = ConsoleSession
812025-04-30T06:43:33.868ZINFOtest_unauthorized (ServerContext): registering Oso class class = DeviceAuthRequest
822025-04-30T06:43:33.868ZINFOtest_unauthorized (ServerContext): registering Oso class class = DeviceAccessToken
832025-04-30T06:43:33.868ZINFOtest_unauthorized (ServerContext): registering Oso class class = PhysicalDisk
842025-04-30T06:43:33.868ZINFOtest_unauthorized (ServerContext): registering Oso class class = Rack
852025-04-30T06:43:33.868ZINFOtest_unauthorized (ServerContext): registering Oso class class = RoleBuiltin
862025-04-30T06:43:33.868ZINFOtest_unauthorized (ServerContext): registering Oso class class = SshKey
872025-04-30T06:43:33.868ZINFOtest_unauthorized (ServerContext): registering Oso class class = Silo
882025-04-30T06:43:33.868ZINFOtest_unauthorized (ServerContext): registering Oso class class = SiloUser
892025-04-30T06:43:33.868ZINFOtest_unauthorized (ServerContext): registering Oso class class = SiloGroup
902025-04-30T06:43:33.868ZINFOtest_unauthorized (ServerContext): registering Oso class class = SupportBundle
912025-04-30T06:43:33.868ZINFOtest_unauthorized (ServerContext): registering Oso class class = IdentityProvider
922025-04-30T06:43:33.868ZINFOtest_unauthorized (ServerContext): registering Oso class class = SamlIdentityProvider
932025-04-30T06:43:33.868ZINFOtest_unauthorized (ServerContext): registering Oso class class = Sled
942025-04-30T06:43:33.868ZINFOtest_unauthorized (ServerContext): registering Oso class class = TufRepo
952025-04-30T06:43:33.868ZINFOtest_unauthorized (ServerContext): registering Oso class class = TufArtifact
962025-04-30T06:43:33.868ZINFOtest_unauthorized (ServerContext): registering Oso class class = WebhookEvent
972025-04-30T06:43:33.868ZINFOtest_unauthorized (ServerContext): registering Oso class class = WebhookReceiver
982025-04-30T06:43:33.868ZINFOtest_unauthorized (ServerContext): registering Oso class class = WebhookSecret
992025-04-30T06:43:33.868ZINFOtest_unauthorized (ServerContext): registering Oso class class = Zpool
1002025-04-30T06:43:33.868ZINFOtest_unauthorized (ServerContext): registering Oso class class = Service
1012025-04-30T06:43:33.868ZINFOtest_unauthorized (ServerContext): registering Oso class class = UserBuiltin
1022025-04-30T06:43:33.869ZINFOtest_unauthorized (ServerContext): full Oso configuration config = #\n# Oso configuration for Omicron\n# This file is augmented by generated snippets.\n#\n\n#\n# ACTOR TYPES AND BASIC RULES\n#\n\n# `AnyActor` includes both authenticated and unauthenticated users.\nactor AnyActor {}\n\n# An `AuthenticatedActor` has an identity in the system. All of our operations\n# today require that an actor be authenticated.\nactor AuthenticatedActor {}\n\n# For any resource, `actor` can perform action `action` on it if they're\n# authenticated and their role(s) give them the corresponding permission on that\n# resource.\nallow(actor: AnyActor, action: Action, resource) if\n actor.authenticated and\n has_permission(actor.authn_actor.unwrap(), action.to_perm(), resource);\n\n# Define role relationships\nhas_role(actor: AuthenticatedActor, role: String, resource: Resource)\n\tif resource.has_role(actor, role);\n\n#\n# ROLES AND PERMISSIONS IN THE FLEET/SILO/PROJECT HIERARCHY\n#\n# We define the following permissions for most resources in the system:\n#\n# - "create_child": required to create child resources (of any type)\n#\n# - "list_children": required to list child resources (of all types) of a\n# resource\n#\n# - "modify": required to modify or delete a resource\n#\n# - "read": required to read a resource\n#\n# We define the following predefined roles for only a few high-level resources:\n# the Fleet (see below), Silo, Organization, and Project. The specific roles\n# are oriented around intended use-cases:\n#\n# - "admin": has all permissions on the resource\n#\n# - "collaborator": has "read", "list_children", and "create_child", plus\n# the "admin" role for child resources. The idea is that if you're an\n# Organization Collaborator, you have full control over the Projects within\n# the Organization, but you cannot modify or delete the Organization itself.\n#\n# - "viewer": has "read" and "list_children" on a resource\n#\n# Below the Project level, permissions are granted via roles at the Project\n# level. For example, for someone to be able to create, modify, or delete any\n# Instances, they must be granted project.collaborator, which means they can\n# create, modify, or delete _all_ resources in the Project.\n#\n# The complete set of predefined roles:\n#\n# - fleet.admin (superuser for the whole system)\n# - fleet.collaborator (can manage Silos)\n# - fleet.viewer (can read most non-siloed resources in the system)\n# - silo.admin (superuser for the silo)\n# - silo.collaborator (can create and own Organizations)\n# - silo.viewer (can read most resources within the Silo)\n# - organization.admin (complete control over an organization)\n# - organization.collaborator (can manage Projects)\n# - organization.viewer (can read most resources within the Organization)\n# - project.admin (complete control over a Project)\n# - project.collaborator (can manage all resources within the Project)\n# - project.viewer (can read most resources within the Project)\n#\n# Outside the Silo/Organization/Project hierarchy, we (currently) treat most\n# resources as nested under Fleet or else a synthetic resource (see below). We\n# do not yet support role assignments on anything other than Fleet, Silo,\n# Organization, or Project.\n#\n\n# "Fleet" is a global singleton representing the whole system. The name comes\n# from the idea described in RFD 24, but it's not quite right. This probably\n# should be more like "Region" or "AvailabilityZone". The precise boundaries\n# have not yet been figured out.\nresource Fleet {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\troles = [\n\t # Roles that can be attached by users\n\t "admin",\n\t "collaborator",\n\t "viewer",\n\n\t # Internal-only roles\n\t "external-authenticator"\n\t];\n\n\t# Roles implied by other roles on this resource\n\t"viewer" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\t"create_child" if "collaborator";\n\t"modify" if "admin";\n}\n\n# For fleets specifically, roles can be conferred by roles on the user's Silo.\nhas_role(actor: AuthenticatedActor, role: String, _: Fleet) if\n\tsilo_role in actor.confers_fleet_role(role) and\n\thas_role(actor, silo_role, actor.silo.unwrap());\n\nresource Silo {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\troles = [ "admin", "collaborator", "viewer" ];\n\n\t# Roles implied by other roles on this resource\n\t"viewer" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\n\t"create_child" if "collaborator";\n\t"modify" if "admin";\n\n\t# Permissions implied by roles on this resource's parent (Fleet). Fleet\n\t# privileges allow a user to see and potentially administer the Silo,\n\t# but they do not give anyone permission to look at anything inside the\n\t# Silo. To achieve this, we use permission rules here. (If we granted\n\t# Fleet administrators _roles_ on the Silo, then those would cascade\n\t# into the Silo as well.)\n\trelations = { parent_fleet: Fleet };\n\t"read" if "viewer" on "parent_fleet";\n\t"modify" if "collaborator" on "parent_fleet";\n\n\t# external authenticator has to create silo users\n\t"list_children" if "external-authenticator" on "parent_fleet";\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\n\nhas_relation(fleet: Fleet, "parent_fleet", silo: Silo)\n\tif silo.fleet = fleet;\n\n# As a special case, all authenticated users can read their own Silo. That's\n# not quite the same as having the "viewer" role. For example, they cannot list\n# Organizations in the Silo.\n#\n# One reason this is necessary is because if an unprivileged user tries to\n# create an Organization using "POST /organizations", they should get back a 403\n# (which implies they're able to see /organizations, which is essentially seeing\n# the Silo itself) rather than a 404. This behavior isn't a hard constraint\n# (i.e., you could reasonably get a 404 for an API you're not allowed to call).\n# Nor is the implementation (i.e., we could special-case this endpoint somehow).\n# But granting this permission is the simplest way to keep this endpoint's\n# behavior consistent with the rest of the API.\n#\n# This rule is also used to determine if a user can list the identity providers\n# in the Silo (which they should be able to), since that's predicated on being\n# able to read the Silo.\n#\n# It's unclear what else would break if users couldn't see their own Silo.\nhas_permission(actor: AuthenticatedActor, "read", silo: Silo)\n\tif silo in actor.silo;\n\nresource Project {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\troles = [ "admin", "collaborator", "viewer" ];\n\n\t# Roles implied by other roles on this resource\n\t"viewer" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\t"create_child" if "collaborator";\n\t"modify" if "admin";\n\n\t# Roles implied by roles on this resource's parent (Silo)\n\trelations = { parent_silo: Silo };\n\t"admin" if "collaborator" on "parent_silo";\n\t"viewer" if "viewer" on "parent_silo";\n}\nhas_relation(silo: Silo, "parent_silo", project: Project)\n\tif project.silo = silo;\n\n#\n# GENERAL RESOURCES OUTSIDE THE SILO/PROJECT HIERARCHY\n#\n# Many resources use snippets of Polar generated by the `authz_resource!` Rust\n# macro. Some resources require custom Polar code. Those appear here.\n#\n\nresource Certificate {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Fleet-level and silo-level roles both grant privileges on certificates.\n\t"read" if "admin" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"read" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", certificate: Certificate)\n\tif certificate.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", certificate: Certificate)\n\tif certificate.silo.fleet = fleet;\n\nresource SiloUser {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\t# Fleet and Silo administrators can manage a Silo's users. This is one\n\t# of the only areas of Silo configuration that Fleet Administrators have\n\t# permissions on.\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\t"list_children" if "read" on "parent_silo";\n\t"read" if "read" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\t"list_children" if "read" on "parent_fleet";\n\t"read" if "read" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", user: SiloUser)\n\tif user.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", user: SiloUser)\n\tif user.silo.fleet = fleet;\n\n# authenticated actors have all permissions on themselves\nhas_permission(actor: AuthenticatedActor, _perm: String, silo_user: SiloUser)\n if actor.equals_silo_user(silo_user);\n\nhas_permission(actor: AuthenticatedActor, "read", silo_user: SiloUser)\n if silo_user.silo in actor.silo;\n\nresource SiloGroup {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\trelations = { parent_silo: Silo };\n\t"list_children" if "read" on "parent_silo";\n\t"read" if "read" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n}\nhas_relation(silo: Silo, "parent_silo", group: SiloGroup)\n\tif group.silo = silo;\n\nresource SshKey {\n\tpermissions = [ "read", "modify" ];\n\trelations = { silo_user: SiloUser };\n\n\t"read" if "read" on "silo_user";\n\t"modify" if "modify" on "silo_user";\n}\nhas_relation(user: SiloUser, "silo_user", ssh_key: SshKey)\n\tif ssh_key.silo_user = user;\n\nresource IdentityProvider {\n\tpermissions = [\n\t "read",\n\t "modify",\n\t "create_child",\n\t "list_children",\n\t];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Silo-level roles grant privileges on identity providers.\n\t"read" if "viewer" on "parent_silo";\n\t"list_children" if "viewer" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\n\t# Fleet-level roles also grant privileges on identity providers.\n\t"read" if "viewer" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", identity_provider: IdentityProvider)\n\tif identity_provider.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: IdentityProvider)\n\tif collection.silo.fleet = fleet;\n\nresource SamlIdentityProvider {\n\tpermissions = [\n\t "read",\n\t "modify",\n\t "create_child",\n\t "list_children",\n\t];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Silo-level roles grant privileges on identity providers.\n\t"read" if "viewer" on "parent_silo";\n\t"list_children" if "viewer" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\n\t# Fleet-level roles also grant privileges on identity providers.\n\t"read" if "viewer" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", saml_identity_provider: SamlIdentityProvider)\n\tif saml_identity_provider.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SamlIdentityProvider)\n\tif collection.silo.fleet = fleet;\n\n#\n# SYNTHETIC RESOURCES OUTSIDE THE SILO HIERARCHY\n#\n# The resources here do not correspond to anything that appears explicitly in\n# the API or is stored in the database. These are used either at the top level\n# of the API path (e.g., "/v1/system/ip-pools") or as an implementation detail of the system\n# (in the case of console sessions and "Database"). The policies are\n# either statically-defined in this file or driven by role assignments on the\n# Fleet. None of these resources defines their own roles.\n#\n\n# Describes the policy for reading and modifying DNS configuration\n# (both internal and external)\nresource DnsConfig {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_fleet: Fleet };\n\t# "external-authenticator" requires these permissions because that's the\n\t# context that Nexus uses when creating and deleting Silos. These\n\t# operations necessarily need to read and modify DNS configuration.\n\t"read" if "external-authenticator" on "parent_fleet";\n\t"modify" if "external-authenticator" on "parent_fleet";\n\t# "admin" on the parent fleet also gets these permissions, primarily for\n\t# the test suite.\n\t"read" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", dns_config: DnsConfig)\n\tif dns_config.fleet = fleet;\n\n# Describes the policy for accessing blueprints\nresource BlueprintConfig {\n\tpermissions = [\n\t "list_children", # list blueprints\n\t "create_child", # create blueprint\n\t "read", # read the current target\n\t "modify", # change the current target\n\t];\n\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"read" if "viewer" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", list: BlueprintConfig)\n\tif list.fleet = fleet;\n\n# Describes the policy for accessing blueprints\nresource TargetReleaseConfig {\n\tpermissions = [\n\t "read", # read the current target release\n\t "modify", # change the current target release\n\t];\n\n\trelations = { parent_fleet: Fleet };\n\t"read" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", resource: TargetReleaseConfig)\n\tif resource.fleet = fleet;\n\n# Describes the policy for reading and modifying low-level inventory\nresource Inventory {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_fleet: Fleet };\n\t"read" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", inventory: Inventory)\n\tif inventory.fleet = fleet;\n\n# Describes the policy for accessing "/v1/system/ip-pools" in the API\nresource IpPoolList {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "create_child",\n\t];\n\n\t# Fleet Administrators can create or modify the IP Pools list.\n\trelations = { parent_fleet: Fleet };\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n\n\t# Fleet Viewers can list IP Pools\n\t"list_children" if "viewer" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", ip_pool_list: IpPoolList)\n\tif ip_pool_list.fleet = fleet;\n\n# Any authenticated user can create a child of a provided IP Pool.\n# This is necessary to use the pools when provisioning instances.\nhas_permission(actor: AuthenticatedActor, "create_child", ip_pool: IpPool)\n\tif silo in actor.silo and silo.fleet = ip_pool.fleet;\n\n# Describes the policy for creating and managing web console sessions.\nresource ConsoleSessionList {\n\tpermissions = [ "create_child" ];\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", collection: ConsoleSessionList)\n\tif collection.fleet = fleet;\n\n# Describes the policy for creating and managing device authorization requests.\nresource DeviceAuthRequestList {\n\tpermissions = [ "create_child" ];\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", collection: DeviceAuthRequestList)\n\tif collection.fleet = fleet;\n\n# Describes the policy for creating and managing Silo certificates\nresource SiloCertificateList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Both Fleet and Silo administrators can see and modify the Silo's\n\t# certificates.\n\t"list_children" if "admin" on "parent_silo";\n\t"list_children" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloCertificateList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloCertificateList)\n\tif collection.silo.fleet = fleet;\n\n# Describes the policy for creating and managing Silo identity providers\nresource SiloIdentityProviderList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Everyone who can read the Silo (which includes all the users in the\n\t# Silo) can see the identity providers in it.\n\t"list_children" if "read" on "parent_silo";\n\n\t# Fleet and Silo administrators can manage the Silo's identity provider\n\t# configuration. This is one of the only areas of Silo configuration\n\t# that Fleet Administrators have permissions on. This is also one of\n\t# the only cases where we need to look two levels up the hierarchy to\n\t# see if somebody has the right permission. For most other things,\n\t# permissions cascade down the hierarchy so we only need to look at the\n\t# parent.\n\t"create_child" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloIdentityProviderList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloIdentityProviderList)\n\tif collection.silo.fleet = fleet;\n\n# Describes the policy for creating and managing Silo users (mostly intended for\n# API-managed users)\nresource SiloUserList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Everyone who can read the Silo (which includes all the users in the\n\t# Silo) can see the users in it.\n\t"list_children" if "read" on "parent_silo";\n\n\t# Fleet and Silo administrators can manage the Silo's users. This is\n\t# one of the only areas of Silo configuration that Fleet Administrators\n\t# have permissions on. This is also one of the few cases (so far) where\n\t# we need to look two levels up the hierarchy to see if somebody has the\n\t# right permission. For most other things, permissions cascade down the\n\t# hierarchy so we only need to look at the parent.\n\t"create_child" if "admin" on "parent_silo";\n\t"list_children" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloUserList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloUserList)\n\tif collection.silo.fleet = fleet;\n\n# These rules grants the external authenticator role the permissions it needs to\n# read silo users and modify their sessions. This is necessary for login to\n# work.\nhas_permission(actor: AuthenticatedActor, "read", silo: Silo)\n\tif has_role(actor, "external-authenticator", silo.fleet);\nhas_permission(actor: AuthenticatedActor, "read", user: SiloUser)\n\tif has_role(actor, "external-authenticator", user.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", user: SiloUser)\n\tif has_role(actor, "external-authenticator", user.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "read", group: SiloGroup)\n\tif has_role(actor, "external-authenticator", group.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", group: SiloGroup)\n\tif has_role(actor, "external-authenticator", group.silo.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", session: ConsoleSession)\n\tif has_role(actor, "external-authenticator", session.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", session: ConsoleSession)\n\tif has_role(actor, "external-authenticator", session.fleet);\n\n# All authenticated users can read and delete device authn requests because\n# by necessity these operations happen before we've figured out what user (or\n# even Silo) the device auth is associated with. Any user can claim a device\n# auth request with the right user code (that's how it works) -- it's the user\n# code and associated logic that prevents unauthorized access here.\nhas_permission(_actor: AuthenticatedActor, "read", _device_auth: DeviceAuthRequest);\nhas_permission(_actor: AuthenticatedActor, "modify", _device_auth: DeviceAuthRequest);\n\nhas_permission(actor: AuthenticatedActor, "read", device_token: DeviceAccessToken)\n\tif has_role(actor, "external-authenticator", device_token.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", identity_provider: IdentityProvider)\n\tif has_role(actor, "external-authenticator", identity_provider.silo.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", saml_identity_provider: SamlIdentityProvider)\n\tif has_role(actor, "external-authenticator", saml_identity_provider.silo.fleet);\n\n# Describes the policy for who can access the internal database.\nresource Database {\n\tpermissions = [\n\t # "query" is required to perform any query against the database,\n\t # whether a read or write query. This is checked when an operation\n\t # checks out a database connection from the connection pool.\n\t #\n\t # Any authenticated user gets this permission. There's generally\n\t # some other authz check involved in the database query. For\n\t # example, if you're querying the database to "read" a "Project", we\n\t # should also be checking that. So why do we do this at all? It's\n\t # a belt-and-suspenders measure so that if we somehow introduced an\n\t # unauthenticated code path that hits the database, it cannot be\n\t # used to DoS the database because we won't allow the operation to\n\t # make the query. (As long as the code path _is_ authenticated, we\n\t # can use throttling mechanisms to prevent DoS.)\n\t "query",\n\n\t # "modify" is required to populate database data that's delivered\n\t # with the system. It should also be required for schema changes,\n\t # when we support those. This is separate from "query" so that we\n\t # cannot accidentally invoke these code paths from API calls and\n\t # other general functions.\n\t "modify"\n\t];\n}\n\n# All authenticated users have the "query" permission on the database.\nhas_permission(_actor: AuthenticatedActor, "query", _resource: Database);\n\n# The "db-init" user is the only one with the "modify" permission.\nhas_permission(USER_DB_INIT: AuthenticatedActor, "modify", _resource: Database);\nhas_permission(USER_DB_INIT: AuthenticatedActor, "create_child", _resource: IpPoolList);\n# It also has "admin" on the internal silo to populate it with built-in resources.\n# TODO-completeness: actually limit to just internal silo and not all silos\nhas_role(USER_DB_INIT: AuthenticatedActor, "admin", _silo: Silo);\n\n# Allow the internal API admin permissions on all silos.\nhas_role(USER_INTERNAL_API: AuthenticatedActor, "admin", _silo: Silo);\n\nresource WebhookSecret {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_webhook_receiver: WebhookReceiver };\n\n\t"read" if "read" on "parent_webhook_receiver";\n\t"modify" if "modify" on "parent_webhook_receiver";\n}\n\nhas_relation(rx: WebhookReceiver, "parent_webhook_receiver", secret: WebhookSecret)\n\tif secret.webhook_receiver = rx;\n\nresource WebhookEventClassList {\n\tpermissions = [ "list_children" ];\n\trelations = { parent_fleet: Fleet };\n\n\t"list_children" if "viewer" on "parent_fleet";\n}\n\nhas_relation(fleet: Fleet, "parent_fleet", collection: WebhookEventClassList)\n\tif collection.fleet = fleet;\n\n\n\n resource Disk {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Disk)\n if child.project = parent;\n \n\n resource Snapshot {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Snapshot)\n if child.project = parent;\n \n\n resource ProjectImage {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: ProjectImage)\n if child.project = parent;\n \n\n resource AffinityGroup {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: AffinityGroup)\n if child.project = parent;\n \n\n resource AntiAffinityGroup {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: AntiAffinityGroup)\n if child.project = parent;\n \n\n resource Instance {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Instance)\n if child.project = parent;\n \n\n resource IpPool {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: IpPool)\n if child.fleet = fleet;\n \n\n resource InstanceNetworkInterface {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Instance\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InstanceNetworkInterface)\n if has_relation(project, "containing_project", child.instance);\n\n has_relation(parent: Instance, "parent", child: InstanceNetworkInterface)\n if child.instance = parent;\n \n\n resource Vpc {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Vpc)\n if child.project = parent;\n \n\n resource VpcRouter {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Vpc\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: VpcRouter)\n if has_relation(project, "containing_project", child.vpc);\n\n has_relation(parent: Vpc, "parent", child: VpcRouter)\n if child.vpc = parent;\n \n\n resource InternetGateway {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Vpc\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InternetGateway)\n if has_relation(project, "containing_project", child.vpc);\n\n has_relation(parent: Vpc, "parent", child: InternetGateway)\n if child.vpc = parent;\n \n\n resource InternetGatewayIpPool {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: InternetGateway\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InternetGatewayIpPool)\n if has_relation(project, "containing_project", child.internet_gateway);\n\n has_relation(parent: InternetGateway, "parent", child: InternetGatewayIpPool)\n if child.internet_gateway = parent;\n \n\n resource InternetGatewayIpAddress {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: InternetGateway\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InternetGatewayIpAddress)\n if has_relation(project, "containing_project", child.internet_gateway);\n\n has_relation(parent: InternetGateway, "parent", child: InternetGatewayIpAddress)\n if child.internet_gateway = parent;\n \n\n resource RouterRoute {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: VpcRouter\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: RouterRoute)\n if has_relation(project, "containing_project", child.vpc_router);\n\n has_relation(parent: VpcRouter, "parent", child: RouterRoute)\n if child.vpc_router = parent;\n \n\n resource VpcSubnet {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Vpc\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: VpcSubnet)\n if has_relation(project, "containing_project", child.vpc);\n\n has_relation(parent: Vpc, "parent", child: VpcSubnet)\n if child.vpc = parent;\n \n\n resource FloatingIp {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: FloatingIp)\n if child.project = parent;\n \n\n resource Image {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_silo: Silo };\n "list_children" if "viewer" on "containing_silo";\n "read" if "viewer" on "containing_silo";\n "modify" if "collaborator" on "containing_silo";\n "create_child" if "collaborator" on "containing_silo";\n }\n\n has_relation(parent: Silo, "containing_silo", child: Image)\n if child.silo = parent;\n \n\n resource SiloImage {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_silo: Silo };\n "list_children" if "viewer" on "containing_silo";\n "read" if "viewer" on "containing_silo";\n "modify" if "collaborator" on "containing_silo";\n "create_child" if "collaborator" on "containing_silo";\n }\n\n has_relation(parent: Silo, "containing_silo", child: SiloImage)\n if child.silo = parent;\n \n\n resource AddressLot {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: AddressLot)\n if child.fleet = fleet;\n \n\n resource Blueprint {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Blueprint)\n if child.fleet = fleet;\n \n\n resource LoopbackAddress {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: LoopbackAddress)\n if child.fleet = fleet;\n \n\n\n resource ConsoleSession {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: ConsoleSession)\n if child.fleet = fleet;\n \n\n resource DeviceAuthRequest {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: DeviceAuthRequest)\n if child.fleet = fleet;\n \n\n resource DeviceAccessToken {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: DeviceAccessToken)\n if child.fleet = fleet;\n \n\n resource PhysicalDisk {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: PhysicalDisk)\n if child.fleet = fleet;\n \n\n resource Rack {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Rack)\n if child.fleet = fleet;\n \n\n resource RoleBuiltin {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: RoleBuiltin)\n if child.fleet = fleet;\n \n\n\n\n\n\n resource SupportBundle {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: SupportBundle)\n if child.fleet = fleet;\n \n\n\n\n resource Sled {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Sled)\n if child.fleet = fleet;\n \n\n resource TufRepo {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: TufRepo)\n if child.fleet = fleet;\n \n\n resource TufArtifact {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: TufArtifact)\n if child.fleet = fleet;\n \n\n resource WebhookEvent {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: WebhookEvent)\n if child.fleet = fleet;\n \n\n resource WebhookReceiver {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: WebhookReceiver)\n if child.fleet = fleet;\n \n\n\n resource Zpool {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Zpool)\n if child.fleet = fleet;\n \n\n resource Service {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Service)\n if child.fleet = fleet;\n \n\n resource UserBuiltin {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: UserBuiltin)\n if child.fleet = fleet;\n
1032025-04-30T06:43:33.895ZINFOtest_unauthorized (ServerContext): Setting up resolver using DNS address: [::1]:38709
1042025-04-30T06:43:33.895ZINFOtest_unauthorized (DnsResolver): new DNS resolver addresses = [[::1]:38709]
1052025-04-30T06:43:33.895ZINFOtest_unauthorized (ServerContext): Setting up qorb database pool from a single host url = PostgresConfigWithUrl {\n url_raw: "postgresql://root@[::1]:46222/omicron?sslmode=disable",\n config: Config {\n user: Some(\n "root",\n ),\n password: None,\n dbname: Some(\n "omicron",\n ),\n options: None,\n application_name: None,\n ssl_mode: Disable,\n host: [\n Tcp(\n "::1",\n ),\n ],\n hostaddr: [],\n port: [\n 46222,\n ],\n connect_timeout: None,\n tcp_user_timeout: None,\n keepalives: true,\n keepalives_idle: 7200s,\n keepalives_interval: None,\n keepalives_retries: None,\n target_session_attrs: Any,\n channel_binding: Prefer,\n load_balance_hosts: Disable,\n },\n}
1062025-04-30T06:43:33.897ZDEBGtest_unauthorized (ServerContext): registered USDT probes
1072025-04-30T06:43:33.937ZINFOtest_unauthorized (nexus): Database schema version is up to date desired_version = 140.0.0 found_version = 140.0.0
1082025-04-30T06:43:34.073ZDEBGtest_unauthorized (clickhouse-client): registered USDT probes id = 841d4277-9027-49d4-9c62-b091eaf98c1d
1092025-04-30T06:43:34.160ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = GET path = /bgtasks versions = all
1102025-04-30T06:43:34.160ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = POST path = /bgtasks/activate versions = all
1112025-04-30T06:43:34.160ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = GET path = /bgtasks/view/{bgtask_name} versions = all
1122025-04-30T06:43:34.160ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = GET path = /clickhouse/policy versions = all
1132025-04-30T06:43:34.160ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = POST path = /clickhouse/policy versions = all
1142025-04-30T06:43:34.160ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = POST path = /crucible/0/upstairs/{upstairs_id}/downstairs/{downstairs_id}/stop-request versions = all
1152025-04-30T06:43:34.160ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = POST path = /crucible/0/upstairs/{upstairs_id}/downstairs/{downstairs_id}/stopped versions = all
1162025-04-30T06:43:34.160ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = POST path = /crucible/0/upstairs/{upstairs_id}/repair/{repair_id}/progress versions = all
1172025-04-30T06:43:34.160ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = POST path = /crucible/0/upstairs/{upstairs_id}/repair-finish versions = all
1182025-04-30T06:43:34.161ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = POST path = /crucible/0/upstairs/{upstairs_id}/repair-start versions = all
1192025-04-30T06:43:34.161ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = POST path = /demo-saga versions = all
1202025-04-30T06:43:34.161ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = POST path = /demo-saga/{demo_saga_id}/complete versions = all
1212025-04-30T06:43:34.161ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = GET path = /deployment/blueprints/all versions = all
1222025-04-30T06:43:34.161ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = DELETE path = /deployment/blueprints/all/{blueprint_id} versions = all
1232025-04-30T06:43:34.161ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = GET path = /deployment/blueprints/all/{blueprint_id} versions = all
1242025-04-30T06:43:34.161ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = POST path = /deployment/blueprints/import versions = all
1252025-04-30T06:43:34.161ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = POST path = /deployment/blueprints/regenerate versions = all
1262025-04-30T06:43:34.161ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = GET path = /deployment/blueprints/target versions = all
1272025-04-30T06:43:34.161ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = POST path = /deployment/blueprints/target versions = all
1282025-04-30T06:43:34.161ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = PUT path = /deployment/blueprints/target/enabled versions = all
1292025-04-30T06:43:34.161ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = POST path = /disk/{disk_id}/remove-read-only-parent versions = all
1302025-04-30T06:43:34.161ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = PUT path = /disks/{disk_id} versions = all
1312025-04-30T06:43:34.161ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = GET path = /experimental/v1/system/support-bundles versions = all
1322025-04-30T06:43:34.161ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = POST path = /experimental/v1/system/support-bundles versions = all
1332025-04-30T06:43:34.161ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = DELETE path = /experimental/v1/system/support-bundles/{support_bundle} versions = all
1342025-04-30T06:43:34.161ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = GET path = /experimental/v1/system/support-bundles/{support_bundle} versions = all
1352025-04-30T06:43:34.161ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = GET path = /experimental/v1/system/support-bundles/{support_bundle}/download versions = all
1362025-04-30T06:43:34.161ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = HEAD path = /experimental/v1/system/support-bundles/{support_bundle}/download versions = all
1372025-04-30T06:43:34.161ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = GET path = /experimental/v1/system/support-bundles/{support_bundle}/download/{file} versions = all
1382025-04-30T06:43:34.161ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = HEAD path = /experimental/v1/system/support-bundles/{support_bundle}/download/{file} versions = all
1392025-04-30T06:43:34.161ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = GET path = /experimental/v1/system/support-bundles/{support_bundle}/index versions = all
1402025-04-30T06:43:34.161ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = POST path = /instances/{instance_id}/migrate versions = all
1412025-04-30T06:43:34.161ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = POST path = /metrics/collectors versions = all
1422025-04-30T06:43:34.161ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = GET path = /metrics/collectors/{collector_id}/producers versions = all
1432025-04-30T06:43:34.161ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = POST path = /metrics/producers versions = all
1442025-04-30T06:43:34.161ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = GET path = /mgs-updates versions = all
1452025-04-30T06:43:34.161ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = GET path = /nat/ipv4/changeset/{from_gen} versions = all
1462025-04-30T06:43:34.161ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = GET path = /oximeter/read-policy versions = all
1472025-04-30T06:43:34.161ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = POST path = /oximeter/read-policy versions = all
1482025-04-30T06:43:34.162ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = POST path = /physical-disk/expunge versions = all
1492025-04-30T06:43:34.162ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = GET path = /probes/{sled} versions = all
1502025-04-30T06:43:34.162ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = PUT path = /racks/{rack_id}/initialization-complete versions = all
1512025-04-30T06:43:34.162ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = GET path = /sagas versions = all
1522025-04-30T06:43:34.162ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = GET path = /sagas/{saga_id} versions = all
1532025-04-30T06:43:34.162ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = GET path = /sled-agents/{sled_id} versions = all
1542025-04-30T06:43:34.162ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = POST path = /sled-agents/{sled_id} versions = all
1552025-04-30T06:43:34.162ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = POST path = /sled-agents/{sled_id}/firewall-rules-update versions = all
1562025-04-30T06:43:34.162ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = POST path = /sleds/add versions = all
1572025-04-30T06:43:34.162ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = POST path = /sleds/expunge versions = all
1582025-04-30T06:43:34.162ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = GET path = /sleds/uninitialized versions = all
1592025-04-30T06:43:34.162ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = PUT path = /switch/{switch_id} versions = all
1602025-04-30T06:43:34.162ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = GET path = /v1/ping versions = all
1612025-04-30T06:43:34.162ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = PUT path = /vmms/{propolis_id} versions = all
1622025-04-30T06:43:34.162ZDEBGtest_unauthorized (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34029 method = POST path = /volume/{volume_id}/remove-read-only-parent versions = all
1632025-04-30T06:43:34.162ZINFOtest_unauthorized (dropshot_internal): listening local_addr = 127.0.0.1:34029
1642025-04-30T06:43:34.162ZDEBGtest_unauthorized (dropshot_internal): successfully registered DTrace USDT probes local_addr = 127.0.0.1:34029
1652025-04-30T06:43:34.162ZINFOtest_unauthorized (SEC): SEC running sec_id = 8d752adb-08b9-405f-a11b-66019bf447b5
1662025-04-30T06:43:34.163ZTRACtest_unauthorized (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1672025-04-30T06:43:34.164ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
1682025-04-30T06:43:34.167ZDEBGtest_unauthorized (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1692025-04-30T06:43:34.168ZDEBGtest_unauthorized (DataLoader): attempting to create built-in users
1702025-04-30T06:43:34.169ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1712025-04-30T06:43:34.171ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
1722025-04-30T06:43:34.172ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1732025-04-30T06:43:34.172ZINFOtest_unauthorized (MgsUpdateDriver): starting MgsUpdateDriver
1742025-04-30T06:43:34.179ZTRACtest_unauthorized (dns): buffer buffer = Length: 79 (0x4f) bytes\n0000: 40 2d 01 00 00 01 00 00 00 00 00 01 10 5f 63 72 @-..........._cr\n0010: 75 63 69 62 6c 65 2d 70 61 6e 74 72 79 04 5f 74 ucible-pantry._t\n0020: 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 cp.control-plane\n0030: 05 6f 78 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 .oxide.internal.\n0040: 00 21 00 01 00 00 29 04 d0 00 00 00 00 00 00 .!....)........ kind = dns peer_addr = [::1]:49711 req_id = 3e617908-5060-4ec2-8561-b997db75869b
1752025-04-30T06:43:34.179ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 16429,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_crucible-pantry._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_crucible-pantry._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 16,\n 95,\n 99,\n 114,\n 117,\n 99,\n 105,\n 98,\n 108,\n 101,\n 45,\n 112,\n 97,\n 110,\n 116,\n 114,\n 121,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:49711 req_id = 3e617908-5060-4ec2-8561-b997db75869b
1762025-04-30T06:43:34.181ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 16429, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_crucible-pantry._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_crucible-pantry._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 16, 95, 99, 114, 117, 99, 105, 98, 108, 101, 45, 112, 97, 110, 116, 114, 121, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_crucible-pantry._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:49711 req_id = 3e617908-5060-4ec2-8561-b997db75869b
1772025-04-30T06:43:34.223ZINFOtest_unauthorized (DataLoader): created 5 built-in users
1782025-04-30T06:43:34.223ZTRACtest_unauthorized (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1792025-04-30T06:43:34.223ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
1802025-04-30T06:43:34.226ZDEBGtest_unauthorized (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1812025-04-30T06:43:34.227ZDEBGtest_unauthorized (DataLoader): attempting to create built-in roles
1822025-04-30T06:43:34.227ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1832025-04-30T06:43:34.227ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
1842025-04-30T06:43:34.227ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1852025-04-30T06:43:34.242ZINFOtest_unauthorized (DataLoader): created 10 built-in roles
1862025-04-30T06:43:34.242ZTRACtest_unauthorized (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1872025-04-30T06:43:34.242ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
1882025-04-30T06:43:34.244ZDEBGtest_unauthorized (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1892025-04-30T06:43:34.244ZDEBGtest_unauthorized (DataLoader): attempting to create built-in role assignments
1902025-04-30T06:43:34.244ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1912025-04-30T06:43:34.244ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
1922025-04-30T06:43:34.244ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1932025-04-30T06:43:34.567ZINFOtest_unauthorized (DataLoader): created 4 built-in role assignments
1942025-04-30T06:43:34.567ZTRACtest_unauthorized (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1952025-04-30T06:43:34.567ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
1962025-04-30T06:43:34.570ZDEBGtest_unauthorized (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1972025-04-30T06:43:34.570ZDEBGtest_unauthorized (DataLoader): attempting to create built-in silos
1982025-04-30T06:43:34.570ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1992025-04-30T06:43:34.570ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
2002025-04-30T06:43:34.570ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2012025-04-30T06:43:34.665ZINFOtest_unauthorized (DataLoader): created 2 built-in silos
2022025-04-30T06:43:34.665ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2032025-04-30T06:43:34.665ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
2042025-04-30T06:43:34.665ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2052025-04-30T06:43:34.684ZTRACtest_unauthorized (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2062025-04-30T06:43:34.684ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
2072025-04-30T06:43:34.686ZDEBGtest_unauthorized (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2082025-04-30T06:43:34.686ZDEBGtest_unauthorized (DataLoader): attempting to create built-in projects
2092025-04-30T06:43:34.686ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2102025-04-30T06:43:34.687ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
2112025-04-30T06:43:34.687ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2122025-04-30T06:43:34.696ZTRACtest_unauthorized (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }
2132025-04-30T06:43:34.696ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
2142025-04-30T06:43:34.696ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2152025-04-30T06:43:34.697ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
2162025-04-30T06:43:34.697ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2172025-04-30T06:43:34.738ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
2182025-04-30T06:43:34.738ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2192025-04-30T06:43:34.738ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
2202025-04-30T06:43:34.739ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2212025-04-30T06:43:34.742ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
2222025-04-30T06:43:34.753ZDEBGtest_unauthorized (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) } result = Ok(())
2232025-04-30T06:43:34.753ZTRACtest_unauthorized (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }
2242025-04-30T06:43:34.753ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
2252025-04-30T06:43:34.753ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2262025-04-30T06:43:34.753ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
2272025-04-30T06:43:34.754ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2282025-04-30T06:43:34.757ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
2292025-04-30T06:43:34.758ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2302025-04-30T06:43:34.758ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
2312025-04-30T06:43:34.758ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2322025-04-30T06:43:34.761ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
2332025-04-30T06:43:34.765ZDEBGtest_unauthorized (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) } result = Ok(())
2342025-04-30T06:43:34.765ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2352025-04-30T06:43:34.765ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
2362025-04-30T06:43:34.766ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2372025-04-30T06:43:34.787ZINFOtest_unauthorized (DataLoader): created built-in services project
2382025-04-30T06:43:34.787ZTRACtest_unauthorized (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2392025-04-30T06:43:34.787ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
2402025-04-30T06:43:34.790ZDEBGtest_unauthorized (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2412025-04-30T06:43:34.790ZDEBGtest_unauthorized (DataLoader): attempting to create built-in VPCs
2422025-04-30T06:43:34.790ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2432025-04-30T06:43:34.790ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
2442025-04-30T06:43:34.790ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2452025-04-30T06:43:34.794ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2462025-04-30T06:43:34.794ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
2472025-04-30T06:43:34.794ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2482025-04-30T06:43:34.797ZTRACtest_unauthorized (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }
2492025-04-30T06:43:34.797ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
2502025-04-30T06:43:34.798ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2512025-04-30T06:43:34.798ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
2522025-04-30T06:43:34.805ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2532025-04-30T06:43:34.812ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
2542025-04-30T06:43:34.812ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2552025-04-30T06:43:34.812ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
2562025-04-30T06:43:34.813ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2572025-04-30T06:43:34.816ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
2582025-04-30T06:43:34.816ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2592025-04-30T06:43:34.816ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
2602025-04-30T06:43:34.817ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2612025-04-30T06:43:34.820ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
2622025-04-30T06:43:34.825ZDEBGtest_unauthorized (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) } result = Ok(())
2632025-04-30T06:43:34.825ZTRACtest_unauthorized (DataLoader): successful lookup of siloed resource "Project" using built-in user
2642025-04-30T06:43:34.826ZTRACtest_unauthorized (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }
2652025-04-30T06:43:34.826ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
2662025-04-30T06:43:34.826ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2672025-04-30T06:43:34.826ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
2682025-04-30T06:43:34.826ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2692025-04-30T06:43:34.828ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
2702025-04-30T06:43:34.828ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2712025-04-30T06:43:34.828ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
2722025-04-30T06:43:34.830ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2732025-04-30T06:43:34.835ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
2742025-04-30T06:43:34.835ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2752025-04-30T06:43:34.835ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
2762025-04-30T06:43:34.836ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2772025-04-30T06:43:34.840ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
2782025-04-30T06:43:34.846ZDEBGtest_unauthorized (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) } result = Ok(())
2792025-04-30T06:43:34.846ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2802025-04-30T06:43:34.846ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
2812025-04-30T06:43:34.846ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2822025-04-30T06:43:34.887ZTRACtest_unauthorized (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }
2832025-04-30T06:43:34.887ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
2842025-04-30T06:43:34.887ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2852025-04-30T06:43:34.887ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
2862025-04-30T06:43:34.888ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2872025-04-30T06:43:34.893ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
2882025-04-30T06:43:34.893ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2892025-04-30T06:43:34.893ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
2902025-04-30T06:43:34.898ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2912025-04-30T06:43:34.901ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
2922025-04-30T06:43:34.901ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2932025-04-30T06:43:34.901ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
2942025-04-30T06:43:34.907ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2952025-04-30T06:43:34.914ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
2962025-04-30T06:43:34.931ZDEBGtest_unauthorized (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") } result = Ok(())
2972025-04-30T06:43:34.932ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2982025-04-30T06:43:34.932ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
2992025-04-30T06:43:34.932ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3002025-04-30T06:43:34.954ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3012025-04-30T06:43:34.954ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
3022025-04-30T06:43:34.955ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3032025-04-30T06:43:34.973ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3042025-04-30T06:43:34.973ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
3052025-04-30T06:43:34.974ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3062025-04-30T06:43:34.987ZTRACtest_unauthorized (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }
3072025-04-30T06:43:34.987ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
3082025-04-30T06:43:34.987ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3092025-04-30T06:43:34.988ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
3102025-04-30T06:43:34.991ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3112025-04-30T06:43:34.996ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
3122025-04-30T06:43:34.996ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3132025-04-30T06:43:34.996ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
3142025-04-30T06:43:34.996ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3152025-04-30T06:43:34.998ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
3162025-04-30T06:43:34.999ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3172025-04-30T06:43:35.001ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
3182025-04-30T06:43:35.003ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3192025-04-30T06:43:35.008ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
3202025-04-30T06:43:35.016ZDEBGtest_unauthorized (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") } result = Ok(())
3212025-04-30T06:43:35.016ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3222025-04-30T06:43:35.016ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
3232025-04-30T06:43:35.016ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3242025-04-30T06:43:35.074ZTRACtest_unauthorized (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) }
3252025-04-30T06:43:35.074ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
3262025-04-30T06:43:35.074ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3272025-04-30T06:43:35.074ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
3282025-04-30T06:43:35.076ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3292025-04-30T06:43:35.079ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
3302025-04-30T06:43:35.079ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3312025-04-30T06:43:35.079ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
3322025-04-30T06:43:35.083ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3332025-04-30T06:43:35.093ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
3342025-04-30T06:43:35.093ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3352025-04-30T06:43:35.093ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
3362025-04-30T06:43:35.094ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3372025-04-30T06:43:35.096ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
3382025-04-30T06:43:35.108ZDEBGtest_unauthorized (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
3392025-04-30T06:43:35.109ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3402025-04-30T06:43:35.109ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
3412025-04-30T06:43:35.109ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3422025-04-30T06:43:35.213ZTRACtest_unauthorized (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) }
3432025-04-30T06:43:35.213ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
3442025-04-30T06:43:35.213ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3452025-04-30T06:43:35.213ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
3462025-04-30T06:43:35.214ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3472025-04-30T06:43:35.219ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
3482025-04-30T06:43:35.220ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3492025-04-30T06:43:35.220ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
3502025-04-30T06:43:35.221ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3512025-04-30T06:43:35.224ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
3522025-04-30T06:43:35.224ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3532025-04-30T06:43:35.224ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
3542025-04-30T06:43:35.225ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3552025-04-30T06:43:35.232ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
3562025-04-30T06:43:35.246ZDEBGtest_unauthorized (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
3572025-04-30T06:43:35.246ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3582025-04-30T06:43:35.246ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
3592025-04-30T06:43:35.247ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3602025-04-30T06:43:35.262ZDEBGtest_unauthorized (DataLoader): attempting to create built-in VPC firewall rules
3612025-04-30T06:43:35.263ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3622025-04-30T06:43:35.264ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
3632025-04-30T06:43:35.265ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3642025-04-30T06:43:35.272ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3652025-04-30T06:43:35.272ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
3662025-04-30T06:43:35.273ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3672025-04-30T06:43:35.277ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3682025-04-30T06:43:35.277ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
3692025-04-30T06:43:35.278ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3702025-04-30T06:43:35.282ZTRACtest_unauthorized (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
3712025-04-30T06:43:35.282ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
3722025-04-30T06:43:35.282ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3732025-04-30T06:43:35.283ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
3742025-04-30T06:43:35.283ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3752025-04-30T06:43:35.287ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
3762025-04-30T06:43:35.288ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3772025-04-30T06:43:35.288ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
3782025-04-30T06:43:35.288ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3792025-04-30T06:43:35.431ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
3802025-04-30T06:43:35.431ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3812025-04-30T06:43:35.431ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
3822025-04-30T06:43:35.432ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3832025-04-30T06:43:35.444ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
3842025-04-30T06:43:35.451ZDEBGtest_unauthorized (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
3852025-04-30T06:43:35.451ZTRACtest_unauthorized (DataLoader): successful lookup of siloed resource "Vpc" using built-in user
3862025-04-30T06:43:35.451ZTRACtest_unauthorized (DataLoader): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
3872025-04-30T06:43:35.451ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
3882025-04-30T06:43:35.451ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3892025-04-30T06:43:35.451ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
3902025-04-30T06:43:35.452ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3912025-04-30T06:43:35.469ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
3922025-04-30T06:43:35.469ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3932025-04-30T06:43:35.469ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
3942025-04-30T06:43:35.470ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3952025-04-30T06:43:35.481ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
3962025-04-30T06:43:35.481ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3972025-04-30T06:43:35.481ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
3982025-04-30T06:43:35.482ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3992025-04-30T06:43:35.494ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
4002025-04-30T06:43:35.508ZDEBGtest_unauthorized (DataLoader): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
4012025-04-30T06:43:35.508ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4022025-04-30T06:43:35.508ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
4032025-04-30T06:43:35.535ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4042025-04-30T06:43:35.566ZTRACtest_unauthorized (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
4052025-04-30T06:43:35.566ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
4062025-04-30T06:43:35.566ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4072025-04-30T06:43:35.566ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
4082025-04-30T06:43:35.566ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4092025-04-30T06:43:35.571ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
4102025-04-30T06:43:35.571ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4112025-04-30T06:43:35.571ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
4122025-04-30T06:43:35.571ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4132025-04-30T06:43:35.594ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
4142025-04-30T06:43:35.594ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4152025-04-30T06:43:35.594ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
4162025-04-30T06:43:35.594ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4172025-04-30T06:43:35.597ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
4182025-04-30T06:43:35.603ZDEBGtest_unauthorized (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
4192025-04-30T06:43:35.603ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4202025-04-30T06:43:35.603ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
4212025-04-30T06:43:35.604ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4222025-04-30T06:43:35.674ZINFOtest_unauthorized (DataLoader): created built-in VPC firewall rules
4232025-04-30T06:43:35.674ZDEBGtest_unauthorized (DataLoader): attempting to create built-in VPC Subnets
4242025-04-30T06:43:35.674ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4252025-04-30T06:43:35.674ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
4262025-04-30T06:43:35.675ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4272025-04-30T06:43:35.679ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4282025-04-30T06:43:35.679ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
4292025-04-30T06:43:35.680ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4302025-04-30T06:43:35.682ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4312025-04-30T06:43:35.682ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
4322025-04-30T06:43:35.683ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4332025-04-30T06:43:35.685ZTRACtest_unauthorized (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
4342025-04-30T06:43:35.685ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
4352025-04-30T06:43:35.685ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4362025-04-30T06:43:35.685ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
4372025-04-30T06:43:35.685ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4382025-04-30T06:43:35.687ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
4392025-04-30T06:43:35.687ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4402025-04-30T06:43:35.687ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
4412025-04-30T06:43:35.688ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4422025-04-30T06:43:35.690ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
4432025-04-30T06:43:35.690ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4442025-04-30T06:43:35.690ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
4452025-04-30T06:43:35.690ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4462025-04-30T06:43:35.692ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
4472025-04-30T06:43:35.698ZDEBGtest_unauthorized (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
4482025-04-30T06:43:35.698ZTRACtest_unauthorized (DataLoader): successful lookup of siloed resource "Vpc" using built-in user
4492025-04-30T06:43:35.698ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4502025-04-30T06:43:35.698ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
4512025-04-30T06:43:35.699ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4522025-04-30T06:43:35.705ZTRACtest_unauthorized (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
4532025-04-30T06:43:35.706ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
4542025-04-30T06:43:35.706ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4552025-04-30T06:43:35.706ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
4562025-04-30T06:43:35.706ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4572025-04-30T06:43:35.708ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
4582025-04-30T06:43:35.708ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4592025-04-30T06:43:35.708ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
4602025-04-30T06:43:35.708ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4612025-04-30T06:43:35.725ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
4622025-04-30T06:43:35.725ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4632025-04-30T06:43:35.725ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
4642025-04-30T06:43:35.725ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4652025-04-30T06:43:35.727ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
4662025-04-30T06:43:35.732ZDEBGtest_unauthorized (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
4672025-04-30T06:43:35.740ZTRACtest_unauthorized (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) }
4682025-04-30T06:43:35.740ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
4692025-04-30T06:43:35.740ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4702025-04-30T06:43:35.740ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
4712025-04-30T06:43:35.741ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4722025-04-30T06:43:35.743ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
4732025-04-30T06:43:35.743ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4742025-04-30T06:43:35.743ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
4752025-04-30T06:43:35.744ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4762025-04-30T06:43:35.746ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
4772025-04-30T06:43:35.746ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4782025-04-30T06:43:35.746ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
4792025-04-30T06:43:35.747ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4802025-04-30T06:43:35.750ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
4812025-04-30T06:43:35.757ZDEBGtest_unauthorized (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
4822025-04-30T06:43:35.757ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4832025-04-30T06:43:35.757ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
4842025-04-30T06:43:35.757ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4852025-04-30T06:43:35.775ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4862025-04-30T06:43:35.776ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
4872025-04-30T06:43:35.776ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4882025-04-30T06:43:35.790ZTRACtest_unauthorized (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
4892025-04-30T06:43:35.790ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
4902025-04-30T06:43:35.790ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4912025-04-30T06:43:35.790ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
4922025-04-30T06:43:35.791ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4932025-04-30T06:43:35.802ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
4942025-04-30T06:43:35.802ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4952025-04-30T06:43:35.803ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
4962025-04-30T06:43:35.804ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4972025-04-30T06:43:35.810ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
4982025-04-30T06:43:35.811ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4992025-04-30T06:43:35.811ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
5002025-04-30T06:43:35.820ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5012025-04-30T06:43:35.828ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
5022025-04-30T06:43:35.837ZDEBGtest_unauthorized (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
5032025-04-30T06:43:35.845ZTRACtest_unauthorized (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) }
5042025-04-30T06:43:35.846ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
5052025-04-30T06:43:35.846ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5062025-04-30T06:43:35.846ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
5072025-04-30T06:43:35.847ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5082025-04-30T06:43:35.877ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
5092025-04-30T06:43:35.878ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5102025-04-30T06:43:35.878ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
5112025-04-30T06:43:35.878ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5122025-04-30T06:43:35.883ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
5132025-04-30T06:43:35.883ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5142025-04-30T06:43:35.883ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
5152025-04-30T06:43:35.886ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5162025-04-30T06:43:35.888ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
5172025-04-30T06:43:35.912ZDEBGtest_unauthorized (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
5182025-04-30T06:43:35.912ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5192025-04-30T06:43:35.912ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
5202025-04-30T06:43:35.912ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5212025-04-30T06:43:35.993ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5222025-04-30T06:43:35.993ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
5232025-04-30T06:43:35.994ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5242025-04-30T06:43:36.006ZTRACtest_unauthorized (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
5252025-04-30T06:43:36.006ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
5262025-04-30T06:43:36.006ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5272025-04-30T06:43:36.006ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
5282025-04-30T06:43:36.007ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5292025-04-30T06:43:36.015ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
5302025-04-30T06:43:36.015ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5312025-04-30T06:43:36.015ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
5322025-04-30T06:43:36.016ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5332025-04-30T06:43:36.021ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
5342025-04-30T06:43:36.021ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5352025-04-30T06:43:36.021ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
5362025-04-30T06:43:36.022ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5372025-04-30T06:43:36.024ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
5382025-04-30T06:43:36.030ZDEBGtest_unauthorized (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
5392025-04-30T06:43:36.036ZTRACtest_unauthorized (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) }
5402025-04-30T06:43:36.036ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
5412025-04-30T06:43:36.036ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5422025-04-30T06:43:36.036ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
5432025-04-30T06:43:36.036ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5442025-04-30T06:43:36.038ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
5452025-04-30T06:43:36.038ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5462025-04-30T06:43:36.039ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
5472025-04-30T06:43:36.039ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5482025-04-30T06:43:36.042ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
5492025-04-30T06:43:36.042ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5502025-04-30T06:43:36.042ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
5512025-04-30T06:43:36.043ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5522025-04-30T06:43:36.071ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
5532025-04-30T06:43:36.090ZDEBGtest_unauthorized (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
5542025-04-30T06:43:36.090ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5552025-04-30T06:43:36.090ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
5562025-04-30T06:43:36.091ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5572025-04-30T06:43:36.111ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5582025-04-30T06:43:36.111ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
5592025-04-30T06:43:36.111ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5602025-04-30T06:43:36.125ZINFOtest_unauthorized (DataLoader): created built-in services vpc subnets
5612025-04-30T06:43:36.125ZINFOtest_unauthorized (DataLoader): created built-in services vpc
5622025-04-30T06:43:36.125ZTRACtest_unauthorized (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5632025-04-30T06:43:36.125ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
5642025-04-30T06:43:36.128ZDEBGtest_unauthorized (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5652025-04-30T06:43:36.128ZDEBGtest_unauthorized (DataLoader): attempting to create silo users
5662025-04-30T06:43:36.129ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5672025-04-30T06:43:36.129ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
5682025-04-30T06:43:36.129ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5692025-04-30T06:43:36.143ZINFOtest_unauthorized (DataLoader): created 2 silo users
5702025-04-30T06:43:36.143ZDEBGtest_unauthorized (DataLoader): attempting to create silo user role assignments
5712025-04-30T06:43:36.143ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5722025-04-30T06:43:36.143ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
5732025-04-30T06:43:36.145ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5742025-04-30T06:43:36.154ZINFOtest_unauthorized (DataLoader): created 2 silo user role assignments
5752025-04-30T06:43:36.155ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5762025-04-30T06:43:36.155ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
5772025-04-30T06:43:36.156ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5782025-04-30T06:43:36.164ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5792025-04-30T06:43:36.164ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
5802025-04-30T06:43:36.165ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5812025-04-30T06:43:36.174ZTRACtest_unauthorized (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = IpPoolList
5822025-04-30T06:43:36.176ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
5832025-04-30T06:43:36.176ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5842025-04-30T06:43:36.176ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
5852025-04-30T06:43:36.177ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5862025-04-30T06:43:36.180ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
5872025-04-30T06:43:36.182ZDEBGtest_unauthorized (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = IpPoolList result = Ok(())
5882025-04-30T06:43:36.183ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5892025-04-30T06:43:36.183ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
5902025-04-30T06:43:36.183ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5912025-04-30T06:43:36.195ZTRACtest_unauthorized (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = IpPoolList
5922025-04-30T06:43:36.195ZTRACtest_unauthorized (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
5932025-04-30T06:43:36.195ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5942025-04-30T06:43:36.196ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
5952025-04-30T06:43:36.196ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5962025-04-30T06:43:36.201ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
5972025-04-30T06:43:36.203ZDEBGtest_unauthorized (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = IpPoolList result = Ok(())
5982025-04-30T06:43:36.203ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5992025-04-30T06:43:36.203ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
6002025-04-30T06:43:36.204ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
6012025-04-30T06:43:36.288ZTRACtest_unauthorized (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
6022025-04-30T06:43:36.288ZDEBGtest_unauthorized (DataLoader): roles roles = RoleSet { roles: {} }
6032025-04-30T06:43:36.289ZDEBGtest_unauthorized (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
6042025-04-30T06:43:36.294ZINFOtest_unauthorized (nexus): populate complete
6052025-04-30T06:43:36.294ZINFOtest_unauthorized (nexus): activating background tasks
6062025-04-30T06:43:36.300ZDEBGtest_unauthorized (init_with_steps): Running step start_sled1
6072025-04-30T06:43:36.301ZINFOtest_unauthorized (omicron_sled_agent::sim::Server): setting up sled agent server sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6082025-04-30T06:43:36.348ZINFOtest_unauthorized (SledAgent): created simulated sled agent server = b6d65341-167c-41df-9b5c-41cded99c229 sim_mode = Explicit sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6092025-04-30T06:43:36.348ZINFOtest_unauthorized (omicron_sled_agent::sim::SimulatedUpstairs): registered sled b6d65341-167c-41df-9b5c-41cded99c229 storage
6102025-04-30T06:43:36.349ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 1 reason = Timeout
6112025-04-30T06:43:36.349ZDEBGtest_unauthorized (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
6122025-04-30T06:43:36.349ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
6132025-04-30T06:43:36.349ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6142025-04-30T06:43:36.350ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6152025-04-30T06:43:36.351ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 1 reason = Timeout
6162025-04-30T06:43:36.351ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal
6172025-04-30T06:43:36.351ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 1 reason = Timeout
6182025-04-30T06:43:36.351ZWARNtest_unauthorized (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_internal dns_group = internal reason = no config nor servers
6192025-04-30T06:43:36.351ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 105.28\u{b5}s iteration = 1 status = {"error":"no config nor servers"}
6202025-04-30T06:43:36.351ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 1 reason = Timeout
6212025-04-30T06:43:36.351ZDEBGtest_unauthorized (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
6222025-04-30T06:43:36.351ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
6232025-04-30T06:43:36.351ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6242025-04-30T06:43:36.352ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6252025-04-30T06:43:36.352ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 1 reason = Timeout
6262025-04-30T06:43:36.352ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal
6272025-04-30T06:43:36.352ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 1 reason = Timeout
6282025-04-30T06:43:36.352ZWARNtest_unauthorized (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_external dns_group = external reason = no config nor servers
6292025-04-30T06:43:36.352ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 22.164\u{b5}s iteration = 1 status = {"error":"no config nor servers"}
6302025-04-30T06:43:36.352ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 1 reason = Timeout
6312025-04-30T06:43:36.352ZINFOtest_unauthorized (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-04-30 06:33:36.352330525 UTC
6322025-04-30T06:43:36.352ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database
6332025-04-30T06:43:36.352ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
6342025-04-30T06:43:36.353ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
6352025-04-30T06:43:36.353ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = external_endpoints iteration = 1 reason = Timeout
6362025-04-30T06:43:36.353ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet
6372025-04-30T06:43:36.353ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = external_endpoints resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
6382025-04-30T06:43:36.353ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
6392025-04-30T06:43:36.353ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6402025-04-30T06:43:36.353ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6412025-04-30T06:43:36.353ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 1 reason = Timeout
6422025-04-30T06:43:36.353ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database
6432025-04-30T06:43:36.353ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
6442025-04-30T06:43:36.354ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
6452025-04-30T06:43:36.354ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = bfd_manager iteration = 1 reason = Timeout
6462025-04-30T06:43:36.354ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database
6472025-04-30T06:43:36.354ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
6482025-04-30T06:43:36.354ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
6492025-04-30T06:43:36.354ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = phantom_disks iteration = 1 reason = Timeout
6502025-04-30T06:43:36.354ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = blueprint_loader iteration = 1 reason = Timeout
6512025-04-30T06:43:36.354ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig
6522025-04-30T06:43:36.354ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = blueprint_loader resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
6532025-04-30T06:43:36.354ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database
6542025-04-30T06:43:36.354ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6552025-04-30T06:43:36.355ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6562025-04-30T06:43:36.355ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = blueprint_executor iteration = 1 reason = Timeout
6572025-04-30T06:43:36.355ZWARNtest_unauthorized (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor reason = no blueprint
6582025-04-30T06:43:36.355ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 48.667\u{b5}s iteration = 1 status = {"error":"no blueprint"}
6592025-04-30T06:43:36.355ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 1 reason = Timeout
6602025-04-30T06:43:36.355ZWARNtest_unauthorized (BackgroundTasks): Blueprint execution: skipped background_task = crdb_node_id_collector reason = no blueprint
6612025-04-30T06:43:36.355ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 45.28\u{b5}s iteration = 1 status = {"error":"no blueprint"}
6622025-04-30T06:43:36.355ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 1 reason = Timeout
6632025-04-30T06:43:36.355ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
6642025-04-30T06:43:36.355ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
6652025-04-30T06:43:36.355ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
6662025-04-30T06:43:36.355ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
6672025-04-30T06:43:36.356ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
6682025-04-30T06:43:36.356ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = support_bundle_collector iteration = 1 reason = Timeout
6692025-04-30T06:43:36.356ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet
6702025-04-30T06:43:36.356ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = support_bundle_collector resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
6712025-04-30T06:43:36.356ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database
6722025-04-30T06:43:36.356ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
6732025-04-30T06:43:36.357ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
6742025-04-30T06:43:36.357ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 1 reason = Timeout
6752025-04-30T06:43:36.357ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 20.751\u{b5}s iteration = 1 status = {"error":"task disabled"}
6762025-04-30T06:43:36.357ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 1 reason = Timeout
6772025-04-30T06:43:36.357ZWARNtest_unauthorized (BackgroundTasks): Blueprint rendezvous: skipped background_task = blueprint_rendezvous reason = no blueprint
6782025-04-30T06:43:36.357ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 16.403\u{b5}s iteration = 1 status = {"error":"no blueprint"}
6792025-04-30T06:43:36.357ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 1 reason = Timeout
6802025-04-30T06:43:36.357ZINFOtest_unauthorized (BackgroundTasks): Decommissioned Disk Cleaner disabled background_task = decommissioned_disk_cleaner
6812025-04-30T06:43:36.357ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 77.654\u{b5}s iteration = 1 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
6822025-04-30T06:43:36.357ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 1 reason = Timeout
6832025-04-30T06:43:36.357ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory
6842025-04-30T06:43:36.357ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_zone_nat_tracker resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
6852025-04-30T06:43:36.357ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database
6862025-04-30T06:43:36.357ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6872025-04-30T06:43:36.358ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6882025-04-30T06:43:36.358ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 1 reason = Timeout
6892025-04-30T06:43:36.358ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet
6902025-04-30T06:43:36.358ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = switch_port_config_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
6912025-04-30T06:43:36.358ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database
6922025-04-30T06:43:36.358ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6932025-04-30T06:43:36.358ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6942025-04-30T06:43:36.358ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = v2p_manager iteration = 1 reason = Timeout
6952025-04-30T06:43:36.359ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
6962025-04-30T06:43:36.359ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6972025-04-30T06:43:36.359ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6982025-04-30T06:43:36.359ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = region_replacement iteration = 1 reason = Timeout
6992025-04-30T06:43:36.359ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database
7002025-04-30T06:43:36.359ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
7012025-04-30T06:43:36.360ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
7022025-04-30T06:43:36.361ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = region_replacement_driver iteration = 1 reason = Timeout
7032025-04-30T06:43:36.361ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database
7042025-04-30T06:43:36.361ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
7052025-04-30T06:43:36.361ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7062025-04-30T06:43:36.361ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = instance_watcher iteration = 1 reason = Timeout
7072025-04-30T06:43:36.361ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet
7082025-04-30T06:43:36.361ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = instance_watcher resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
7092025-04-30T06:43:36.361ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database
7102025-04-30T06:43:36.361ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
7112025-04-30T06:43:36.362ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
7122025-04-30T06:43:36.363ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = instance_updater iteration = 1 reason = Timeout
7132025-04-30T06:43:36.363ZINFOtest_unauthorized (BackgroundTasks): background instance updater explicitly disabled background_task = instance_updater
7142025-04-30T06:43:36.363ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = instance_updater elapsed = 479.776\u{b5}s iteration = 1 status = {"destroyed_active_vmms":0,"disabled":true,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
7152025-04-30T06:43:36.363ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = instance_reincarnation iteration = 1 reason = Timeout
7162025-04-30T06:43:36.364ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database
7172025-04-30T06:43:36.364ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
7182025-04-30T06:43:36.364ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
7192025-04-30T06:43:36.364ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 1 reason = Timeout
7202025-04-30T06:43:36.364ZDEBGtest_unauthorized (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
7212025-04-30T06:43:36.364ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
7222025-04-30T06:43:36.364ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7232025-04-30T06:43:36.365ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7242025-04-30T06:43:36.365ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = vpc_route_manager iteration = 1 reason = Timeout
7252025-04-30T06:43:36.365ZINFOtest_unauthorized (BackgroundTasks): VPC route manager running background_task = vpc_route_manager
7262025-04-30T06:43:36.365ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet
7272025-04-30T06:43:36.365ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = vpc_route_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
7282025-04-30T06:43:36.365ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
7292025-04-30T06:43:36.365ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7302025-04-30T06:43:36.366ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7312025-04-30T06:43:36.366ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 1 reason = Timeout
7322025-04-30T06:43:36.366ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database
7332025-04-30T06:43:36.366ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
7342025-04-30T06:43:36.366ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
7352025-04-30T06:43:36.366ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = saga_recovery iteration = 1 reason = Timeout
7362025-04-30T06:43:36.366ZDEBGtest_unauthorized (SagaRecoverer): listing candidate sagas for recovery
7372025-04-30T06:43:36.366ZTRACtest_unauthorized (SagaRecoverer): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000003a8a, .. }) resource = Database
7382025-04-30T06:43:36.366ZDEBGtest_unauthorized (SagaRecoverer): roles roles = RoleSet { roles: {} }
7392025-04-30T06:43:36.367ZDEBGtest_unauthorized (SagaRecoverer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000003a8a, .. }) resource = Database result = Ok(())
7402025-04-30T06:43:36.367ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = lookup_region_port iteration = 1 reason = Timeout
7412025-04-30T06:43:36.368ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database
7422025-04-30T06:43:36.368ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
7432025-04-30T06:43:36.368ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
7442025-04-30T06:43:36.369ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 1 reason = Timeout
7452025-04-30T06:43:36.369ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database
7462025-04-30T06:43:36.369ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
7472025-04-30T06:43:36.369ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
7482025-04-30T06:43:36.369ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 1 reason = Timeout
7492025-04-30T06:43:36.369ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database
7502025-04-30T06:43:36.369ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
7512025-04-30T06:43:36.370ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
7522025-04-30T06:43:36.370ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 1 reason = Timeout
7532025-04-30T06:43:36.370ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database
7542025-04-30T06:43:36.370ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
7552025-04-30T06:43:36.370ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
7562025-04-30T06:43:36.370ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 1 reason = Timeout
7572025-04-30T06:43:36.370ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database
7582025-04-30T06:43:36.370ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
7592025-04-30T06:43:36.371ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
7602025-04-30T06:43:36.371ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = tuf_artifact_replication iteration = 1 reason = Timeout
7612025-04-30T06:43:36.371ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet
7622025-04-30T06:43:36.371ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = tuf_artifact_replication resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
7632025-04-30T06:43:36.371ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database
7642025-04-30T06:43:36.371ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
7652025-04-30T06:43:36.371ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
7662025-04-30T06:43:36.371ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 1 reason = Timeout
7672025-04-30T06:43:36.371ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database
7682025-04-30T06:43:36.371ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
7692025-04-30T06:43:36.373ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
7702025-04-30T06:43:36.373ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = webhook_dispatcher iteration = 1 reason = Timeout
7712025-04-30T06:43:36.373ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_dispatcher resource = Database
7722025-04-30T06:43:36.373ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = webhook_dispatcher roles = RoleSet { roles: {} }
7732025-04-30T06:43:36.373ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_dispatcher resource = Database result = Ok(())
7742025-04-30T06:43:36.373ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = webhook_deliverator iteration = 1 reason = Timeout
7752025-04-30T06:43:36.373ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database
7762025-04-30T06:43:36.373ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
7772025-04-30T06:43:36.373ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
7782025-04-30T06:43:36.373ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig
7792025-04-30T06:43:36.373ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_internal dns_group = internal resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
7802025-04-30T06:43:36.373ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
7812025-04-30T06:43:36.373ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
7822025-04-30T06:43:36.374ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
7832025-04-30T06:43:36.376ZINFOtest_unauthorized (ArtifactStore): No ledger in /var/tmp/omicron_tmp/.tmprM0M8I/artifact-config.json server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
7842025-04-30T06:43:36.376ZDEBGtest_unauthorized (ArtifactStore): Failed to read ledger: Not found in storage path = /var/tmp/omicron_tmp/.tmprM0M8I/artifact-config.json server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
7852025-04-30T06:43:36.376ZINFOtest_unauthorized (ArtifactStore): No ledger in /var/tmp/omicron_tmp/.tmpzboduk/artifact-config.json server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
7862025-04-30T06:43:36.376ZDEBGtest_unauthorized (ArtifactStore): Failed to read ledger: Not found in storage path = /var/tmp/omicron_tmp/.tmpzboduk/artifact-config.json server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
7872025-04-30T06:43:36.449ZDEBGtest_unauthorized (dropshot (Repo Depot)): registered endpoint local_addr = [::1]:48692 method = GET path = /artifact/sha256/{sha256} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
7882025-04-30T06:43:36.449ZINFOtest_unauthorized (dropshot (Repo Depot)): listening local_addr = [::1]:48692 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
7892025-04-30T06:43:36.450ZDEBGtest_unauthorized (dropshot (Repo Depot)): successfully registered DTrace USDT probes local_addr = [::1]:48692 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
7902025-04-30T06:43:36.458ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = GET path = /artifacts sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
7912025-04-30T06:43:36.458ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = PUT path = /artifacts/{sha256} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
7922025-04-30T06:43:36.458ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = POST path = /artifacts/{sha256}/copy-from-depot sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
7932025-04-30T06:43:36.458ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = GET path = /artifacts-config sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
7942025-04-30T06:43:36.458ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = PUT path = /artifacts-config sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
7952025-04-30T06:43:36.458ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = POST path = /boot-disk/{boot_disk}/os/write sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
7962025-04-30T06:43:36.458ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = GET path = /boot-disk/{boot_disk}/os/write/status sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
7972025-04-30T06:43:36.458ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = DELETE path = /boot-disk/{boot_disk}/os/write/status/{update_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
7982025-04-30T06:43:36.458ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = GET path = /bootstore/status sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
7992025-04-30T06:43:36.458ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = GET path = /datasets sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8002025-04-30T06:43:36.458ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = PUT path = /disks/{disk_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8012025-04-30T06:43:36.458ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = POST path = /disks/{disk_id}/poke sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8022025-04-30T06:43:36.458ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = PUT path = /eip-gateways sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8032025-04-30T06:43:36.458ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = GET path = /inventory sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8042025-04-30T06:43:36.458ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = GET path = /network-bootstore-config sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8052025-04-30T06:43:36.458ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = PUT path = /network-bootstore-config sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8062025-04-30T06:43:36.458ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = PUT path = /omicron-config sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8072025-04-30T06:43:36.458ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = GET path = /omicron-physical-disks sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8082025-04-30T06:43:36.458ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = GET path = /sled-identifiers sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8092025-04-30T06:43:36.458ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = GET path = /sled-role sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8102025-04-30T06:43:36.458ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = PUT path = /sleds sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8112025-04-30T06:43:36.458ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = GET path = /support/dladm-info sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8122025-04-30T06:43:36.458ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = GET path = /support/ipadm-info sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8132025-04-30T06:43:36.458ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = GET path = /support/logs/download/{zone} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8142025-04-30T06:43:36.458ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = GET path = /support/logs/zones sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8152025-04-30T06:43:36.458ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = GET path = /support/nvmeadm-info sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8162025-04-30T06:43:36.458ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = GET path = /support/pargs-info sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8172025-04-30T06:43:36.458ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = GET path = /support/pfiles-info sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8182025-04-30T06:43:36.458ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = GET path = /support/pstack-info sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8192025-04-30T06:43:36.458ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = GET path = /support/zfs-info sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8202025-04-30T06:43:36.458ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = GET path = /support/zoneadm-info sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8212025-04-30T06:43:36.458ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = GET path = /support/zpool-info sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8222025-04-30T06:43:36.458ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = GET path = /support-bundles/{zpool_id}/{dataset_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8232025-04-30T06:43:36.458ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = DELETE path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8242025-04-30T06:43:36.458ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = POST path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8252025-04-30T06:43:36.458ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = GET path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id}/download sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8262025-04-30T06:43:36.458ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = HEAD path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id}/download sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8272025-04-30T06:43:36.459ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = GET path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id}/download/{file} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8282025-04-30T06:43:36.459ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = HEAD path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id}/download/{file} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8292025-04-30T06:43:36.459ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = GET path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id}/index sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8302025-04-30T06:43:36.459ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = HEAD path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id}/index sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8312025-04-30T06:43:36.459ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = POST path = /switch-ports sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8322025-04-30T06:43:36.459ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = GET path = /timesync sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8332025-04-30T06:43:36.459ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = DELETE path = /v2p sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8342025-04-30T06:43:36.459ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = GET path = /v2p sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8352025-04-30T06:43:36.459ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = PUT path = /v2p sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8362025-04-30T06:43:36.459ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = DELETE path = /vmms/{propolis_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8372025-04-30T06:43:36.459ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = PUT path = /vmms/{propolis_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8382025-04-30T06:43:36.459ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = POST path = /vmms/{propolis_id}/disks/{disk_id}/snapshot sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8392025-04-30T06:43:36.459ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = DELETE path = /vmms/{propolis_id}/external-ip sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8402025-04-30T06:43:36.459ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = PUT path = /vmms/{propolis_id}/external-ip sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8412025-04-30T06:43:36.459ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = POST path = /vmms/{propolis_id}/poke sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8422025-04-30T06:43:36.459ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = POST path = /vmms/{propolis_id}/poke-single-step sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8432025-04-30T06:43:36.459ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = POST path = /vmms/{propolis_id}/sim-migration-source sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8442025-04-30T06:43:36.459ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = GET path = /vmms/{propolis_id}/state sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8452025-04-30T06:43:36.459ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = PUT path = /vmms/{propolis_id}/state sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8462025-04-30T06:43:36.459ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = PUT path = /vpc/{vpc_id}/firewall/rules sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8472025-04-30T06:43:36.459ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = GET path = /vpc-routes sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8482025-04-30T06:43:36.459ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = PUT path = /vpc-routes sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8492025-04-30T06:43:36.459ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = GET path = /zones sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8502025-04-30T06:43:36.459ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = POST path = /zones/bundle-cleanup sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8512025-04-30T06:43:36.459ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = GET path = /zones/bundle-cleanup/context sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8522025-04-30T06:43:36.459ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = PUT path = /zones/bundle-cleanup/context sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8532025-04-30T06:43:36.459ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = GET path = /zones/bundle-cleanup/utilization sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8542025-04-30T06:43:36.459ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = GET path = /zones/bundles sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8552025-04-30T06:43:36.459ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = GET path = /zones/bundles/{zone_name} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8562025-04-30T06:43:36.459ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = DELETE path = /zones/bundles/{zone_name}/{bundle_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8572025-04-30T06:43:36.459ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = GET path = /zones/bundles/{zone_name}/{bundle_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8582025-04-30T06:43:36.459ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:57786 method = GET path = /zpools sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8592025-04-30T06:43:36.459ZINFOtest_unauthorized (dropshot): listening local_addr = [::1]:57786 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
8602025-04-30T06:43:36.465ZDEBGtest_unauthorized (dropshot): successfully registered DTrace USDT probes local_addr = [::1]:57786 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
8612025-04-30T06:43:36.466ZDEBGtest_unauthorized (omicron_sled_agent::sim::Server): contacting server nexus sled_id = b6d65341-167c-41df-9b5c-41cded99c229
8622025-04-30T06:43:36.466ZDEBGtest_unauthorized (NexusClient): client request body = Some(Body) method = POST sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = http://127.0.0.1:34029/sled-agents/b6d65341-167c-41df-9b5c-41cded99c229
8632025-04-30T06:43:36.467ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig
8642025-04-30T06:43:36.467ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_external dns_group = external resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
8652025-04-30T06:43:36.467ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
8662025-04-30T06:43:36.467ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
8672025-04-30T06:43:36.468ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
8682025-04-30T06:43:36.469ZINFOtest_unauthorized (dropshot_internal): accepted connection local_addr = 127.0.0.1:34029 remote_addr = 127.0.0.1:44231
8692025-04-30T06:43:36.469ZTRACtest_unauthorized (dns): buffer buffer = Length: 75 (0x4b) bytes\n0000: 7a 6c 01 00 00 01 00 00 00 00 00 01 0c 5f 6e 61 zl..........._na\n0010: 6d 65 73 65 72 76 69 63 65 04 5f 74 63 70 0d 63 meservice._tcp.c\n0020: 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 ontrol-plane.oxi\n0030: 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 de.internal..!..\n0040: 00 00 29 04 d0 00 00 00 00 00 00 ..)........ kind = dns peer_addr = [::1]:49194 req_id = 41eb14ea-0093-467b-9b46-32e48f46b753
8702025-04-30T06:43:36.469ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 31340,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_nameservice._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_nameservice._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 12,\n 95,\n 110,\n 97,\n 109,\n 101,\n 115,\n 101,\n 114,\n 118,\n 105,\n 99,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:49194 req_id = 41eb14ea-0093-467b-9b46-32e48f46b753
8712025-04-30T06:43:36.470ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 31340, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_nameservice._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 12, 95, 110, 97, 109, 101, 115, 101, 114, 118, 105, 99, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_nameservice._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:49194 req_id = 41eb14ea-0093-467b-9b46-32e48f46b753
8722025-04-30T06:43:36.470ZTRACtest_unauthorized (dns): buffer buffer = Length: 76 (0x4c) bytes\n0000: 1a 42 01 00 00 01 00 00 00 00 00 01 0d 5f 65 78 .B..........._ex\n0010: 74 65 72 6e 61 6c 2d 64 6e 73 04 5f 74 63 70 0d ternal-dns._tcp.\n0020: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0030: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 21 00 ide.internal..!.\n0040: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:63296 req_id = 075c7f71-44a4-4601-a1d2-4847a88a001e
8732025-04-30T06:43:36.470ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 6722,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_external-dns._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_external-dns._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 13,\n 95,\n 101,\n 120,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 45,\n 100,\n 110,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63296 req_id = 075c7f71-44a4-4601-a1d2-4847a88a001e
8742025-04-30T06:43:36.470ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 6722, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_external-dns._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 13, 95, 101, 120, 116, 101, 114, 110, 97, 108, 45, 100, 110, 115, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_external-dns._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:63296 req_id = 075c7f71-44a4-4601-a1d2-4847a88a001e
8752025-04-30T06:43:36.471ZWARNtest_unauthorized (BackgroundTasks): failed to lookup DNS servers background_task = dns_servers_internal dns_group = internal error = no record found for Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
8762025-04-30T06:43:36.471ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 120.875437ms iteration = 1 status = {"error":"failed to read list of DNS servers: no record found for Query { name: Name(\\"_nameservice._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }"}
8772025-04-30T06:43:36.472ZWARNtest_unauthorized (BackgroundTasks): failed to lookup DNS servers background_task = dns_servers_external dns_group = external error = no record found for Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
8782025-04-30T06:43:36.472ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 119.967959ms iteration = 1 status = {"error":"failed to read list of DNS servers: no record found for Query { name: Name(\\"_external-dns._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }"}
8792025-04-30T06:43:36.472ZTRACtest_unauthorized (dropshot_internal): incoming request local_addr = 127.0.0.1:34029 method = POST remote_addr = 127.0.0.1:44231 req_id = 3e61d2b5-20c2-4954-8337-5cfe7a3dace5 uri = /sled-agents/b6d65341-167c-41df-9b5c-41cded99c229
8802025-04-30T06:43:36.475ZINFOtest_unauthorized (nexus): registered sled agent sled_uuid = b6d65341-167c-41df-9b5c-41cded99c229
8812025-04-30T06:43:36.488ZINFOtest_unauthorized (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-04-30 06:33:36.352330525 UTC pruned = {}
8822025-04-30T06:43:36.488ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 136.61136ms iteration = 1 status = {"expiration":"2025-04-30T06:33:36.352330525Z","pruned":[]}
8832025-04-30T06:43:36.491ZTRACtest_unauthorized (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
8842025-04-30T06:43:36.497ZTRACtest_unauthorized (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
8852025-04-30T06:43:36.499ZTRACtest_unauthorized (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 3a 6c 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 :l..........._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:57728 req_id = 518e415e-aa17-4d46-b452-0aeface87b4c
8862025-04-30T06:43:36.499ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 14956,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:57728 req_id = 518e415e-aa17-4d46-b452-0aeface87b4c
8872025-04-30T06:43:36.501ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 14956, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_dendrite._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 9, 95, 100, 101, 110, 100, 114, 105, 116, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_dendrite._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:57728 req_id = 518e415e-aa17-4d46-b452-0aeface87b4c
8882025-04-30T06:43:36.501ZTRACtest_unauthorized (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 9a d5 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 ............._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:55351 req_id = 1473be36-22a5-4126-bedd-b7620f9539ad
8892025-04-30T06:43:36.501ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 39637,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:55351 req_id = 1473be36-22a5-4126-bedd-b7620f9539ad
8902025-04-30T06:43:36.501ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 39637, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_dendrite._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 9, 95, 100, 101, 110, 100, 114, 105, 116, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_dendrite._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:55351 req_id = 1473be36-22a5-4126-bedd-b7620f9539ad
8912025-04-30T06:43:36.514ZERROtest_unauthorized (BackgroundTasks): failed to resolve addresses for Dendrite services background_task = nat_v4_garbage_collector error = no record found for Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
8922025-04-30T06:43:36.515ZERROtest_unauthorized (BackgroundTasks): failed to resolve addresses for Dendrite services background_task = nat_v4_garbage_collector error = no record found for Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
8932025-04-30T06:43:36.515ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 161.413175ms iteration = 1 status = {"error":"failed to resolve addresses for Dendrite services: no record found for Query { name: Name(\\"_dendrite._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }"}
8942025-04-30T06:43:36.515ZERROtest_unauthorized (BackgroundTasks): failed to resolve addresses for Dendrite services background_task = bfd_manager error = no record found for Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
8952025-04-30T06:43:36.515ZERROtest_unauthorized (BackgroundTasks): failed to resolve addresses for Dendrite services background_task = bfd_manager error = no record found for Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
8962025-04-30T06:43:36.515ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 161.3498ms iteration = 1 status = {"error":"failed to resolve addresses for Dendrite services: no record found for Query { name: Name(\\"_dendrite._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }"}
8972025-04-30T06:43:36.573ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 218.584976ms iteration = 1 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
8982025-04-30T06:43:36.630ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8992025-04-30T06:43:36.650ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9002025-04-30T06:43:36.650ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database
9012025-04-30T06:43:36.650ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9022025-04-30T06:43:36.651ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9032025-04-30T06:43:36.652ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9042025-04-30T06:43:36.657ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
9052025-04-30T06:43:36.657ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database
9062025-04-30T06:43:36.657ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
9072025-04-30T06:43:36.657ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
9082025-04-30T06:43:36.657ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9092025-04-30T06:43:36.662ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
9102025-04-30T06:43:36.662ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
9112025-04-30T06:43:36.662ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
9122025-04-30T06:43:36.662ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
9132025-04-30T06:43:36.663ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9142025-04-30T06:43:36.669ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
9152025-04-30T06:43:36.669ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database
9162025-04-30T06:43:36.669ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9172025-04-30T06:43:36.670ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9182025-04-30T06:43:36.671ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9192025-04-30T06:43:36.678ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
9202025-04-30T06:43:36.678ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database
9212025-04-30T06:43:36.678ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9222025-04-30T06:43:36.679ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9232025-04-30T06:43:36.681ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9242025-04-30T06:43:36.701ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
9252025-04-30T06:43:36.701ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
9262025-04-30T06:43:36.701ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
9272025-04-30T06:43:36.702ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
9282025-04-30T06:43:36.742ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
9292025-04-30T06:43:36.742ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
9302025-04-30T06:43:36.767ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
9312025-04-30T06:43:36.770ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9322025-04-30T06:43:36.828ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
9332025-04-30T06:43:36.829ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet
9342025-04-30T06:43:36.829ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = vpc_route_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
9352025-04-30T06:43:36.829ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
9362025-04-30T06:43:36.830ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9372025-04-30T06:43:36.832ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9382025-04-30T06:43:36.833ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database
9392025-04-30T06:43:36.833ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9402025-04-30T06:43:36.834ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9412025-04-30T06:43:36.847ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9422025-04-30T06:43:36.853ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
9432025-04-30T06:43:36.892ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database
9442025-04-30T06:43:36.896ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
9452025-04-30T06:43:36.897ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
9462025-04-30T06:43:36.898ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database
9472025-04-30T06:43:36.898ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
9482025-04-30T06:43:36.899ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
9492025-04-30T06:43:36.900ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 532.94105ms iteration = 1 status = {"errors":[],"found_port_ok":[]}
9502025-04-30T06:43:36.963ZDEBGtest_unauthorized (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
9512025-04-30T06:43:36.963ZINFOtest_unauthorized (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
9522025-04-30T06:43:36.963ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 597.488116ms iteration = 1 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
9532025-04-30T06:43:36.996ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9542025-04-30T06:43:37.009ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
9552025-04-30T06:43:37.011ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet
9562025-04-30T06:43:37.011ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = tuf_artifact_replication resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
9572025-04-30T06:43:37.011ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database
9582025-04-30T06:43:37.011ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
9592025-04-30T06:43:37.012ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
9602025-04-30T06:43:37.014ZINFOtest_unauthorized (SagaRecoverer): listed in-progress sagas count = 0
9612025-04-30T06:43:37.014ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = saga_recovery elapsed = 648.047412ms iteration = 1 status = {"last_pass":{"Success":{"nfailed":0,"nfound":0,"nrecovered":0,"nremoved":0,"nskipped":0}},"ntotal_failures":0,"ntotal_finished":0,"ntotal_recovered":0,"ntotal_sec_errors_bad_state":0,"ntotal_sec_errors_missing":0,"ntotal_started":0,"recent_failures":[],"recent_recoveries":[]}
9622025-04-30T06:43:37.080ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9632025-04-30T06:43:37.098ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
9642025-04-30T06:43:37.139ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9652025-04-30T06:43:37.153ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
9662025-04-30T06:43:37.167ZTRACtest_unauthorized (BackgroundTasks): no more instances in need of reincarnation background_task = instance_reincarnation reincarnation_reason = instance failed total_found = 0
9672025-04-30T06:43:37.168ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database
9682025-04-30T06:43:37.168ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
9692025-04-30T06:43:37.169ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
9702025-04-30T06:43:37.178ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 807.443278ms iteration = 1 status = {"errors":[],"finish_invoked_ok":[]}
9712025-04-30T06:43:37.178ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 805.752108ms iteration = 1 status = {"by_rx":{},"error":null}
9722025-04-30T06:43:37.195ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 826.300409ms iteration = 1 status = {"errors":[],"garbage_collect_requested":[]}
9732025-04-30T06:43:37.237ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database
9742025-04-30T06:43:37.237ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
9752025-04-30T06:43:37.238ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
9762025-04-30T06:43:37.265ZWARNtest_unauthorized (BackgroundTasks): failed to read target blueprint background_task = blueprint_loader error = Internal Error: no target blueprint set
9772025-04-30T06:43:37.265ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 910.623358ms iteration = 1 status = {"error":"failed to read target blueprint: Internal Error: no target blueprint set"}
9782025-04-30T06:43:37.290ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 918.427176ms iteration = 1 status = {"errors":[],"requests_created_ok":[]}
9792025-04-30T06:43:37.307ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
9802025-04-30T06:43:37.310ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
9812025-04-30T06:43:37.311ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
9822025-04-30T06:43:37.311ZWARNtest_unauthorized (BackgroundTasks): failed to read DNS config background_task = dns_config_internal dns_group = internal error = Internal Error: expected exactly one latest version for DNS group Internal, found 0
9832025-04-30T06:43:37.311ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 962.396322ms iteration = 1 status = {"error":"failed to read DNS config: Internal Error: expected exactly one latest version for DNS group Internal, found 0"}
9842025-04-30T06:43:37.311ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig
9852025-04-30T06:43:37.312ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = external_endpoints resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
9862025-04-30T06:43:37.312ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
9872025-04-30T06:43:37.312ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
9882025-04-30T06:43:37.312ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
9892025-04-30T06:43:37.312ZWARNtest_unauthorized (BackgroundTasks): inventory collection is None background_task = service_zone_nat_tracker
9902025-04-30T06:43:37.312ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 954.886028ms iteration = 1 status = {"error":"inventory collection is None"}
9912025-04-30T06:43:37.321ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database
9922025-04-30T06:43:37.321ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
9932025-04-30T06:43:37.322ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
9942025-04-30T06:43:37.348ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 989.7837ms iteration = 1 status = {}
9952025-04-30T06:43:37.362ZWARNtest_unauthorized (BackgroundTasks): failed to read DNS config background_task = dns_config_external dns_group = external error = Internal Error: expected exactly one latest version for DNS group External, found 0
9962025-04-30T06:43:37.370ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 1.018129807s iteration = 1 status = {"error":"failed to read DNS config: Internal Error: expected exactly one latest version for DNS group External, found 0"}
9972025-04-30T06:43:37.371ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
9982025-04-30T06:43:37.371ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
9992025-04-30T06:43:37.372ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
10002025-04-30T06:43:37.372ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10012025-04-30T06:43:37.375ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
10022025-04-30T06:43:37.375ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
10032025-04-30T06:43:37.375ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10042025-04-30T06:43:37.376ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10052025-04-30T06:43:37.376ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = []
10062025-04-30T06:43:37.376ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
10072025-04-30T06:43:37.377ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: b3 07 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:49426 req_id = 9ecdc2b9-5ade-4cfd-a91f-0ada84fa4bed
10082025-04-30T06:43:37.377ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 45831,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:49426 req_id = 9ecdc2b9-5ade-4cfd-a91f-0ada84fa4bed
10092025-04-30T06:43:37.377ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 45831, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_mgs._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 4, 95, 109, 103, 115, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_mgs._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:49426 req_id = 9ecdc2b9-5ade-4cfd-a91f-0ada84fa4bed
10102025-04-30T06:43:37.378ZWARNtest_unauthorized (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: looking up MGS addresses: no record found for Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
10112025-04-30T06:43:37.378ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 1.022650123s iteration = 1 status = {"error":"failed to collect inventory: looking up MGS addresses: no record found for Query { name: Name(\\"_mgs._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }"}
10122025-04-30T06:43:37.394ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet
10132025-04-30T06:43:37.394ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = support_bundle_collector resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
10142025-04-30T06:43:37.394ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database
10152025-04-30T06:43:37.394ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
10162025-04-30T06:43:37.395ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
10172025-04-30T06:43:37.409ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10182025-04-30T06:43:37.412ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
10192025-04-30T06:43:37.412ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database
10202025-04-30T06:43:37.412ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
10212025-04-30T06:43:37.413ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
10222025-04-30T06:43:37.441ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10232025-04-30T06:43:37.447ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
10242025-04-30T06:43:37.448ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
10252025-04-30T06:43:37.448ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
10262025-04-30T06:43:37.448ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
10272025-04-30T06:43:37.449ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 2 reason = Signaled
10282025-04-30T06:43:37.449ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
10292025-04-30T06:43:37.449ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
10302025-04-30T06:43:37.449ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
10312025-04-30T06:43:37.449ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
10322025-04-30T06:43:37.450ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
10332025-04-30T06:43:37.450ZINFOtest_unauthorized (dropshot_internal): request completed latency_us = 978089 local_addr = 127.0.0.1:34029 method = POST remote_addr = 127.0.0.1:44231 req_id = 3e61d2b5-20c2-4954-8337-5cfe7a3dace5 response_code = 204 uri = /sled-agents/b6d65341-167c-41df-9b5c-41cded99c229
10342025-04-30T06:43:37.452ZDEBGtest_unauthorized (NexusClient): client response result = Ok(Response { url: "http://127.0.0.1:34029/sled-agents/b6d65341-167c-41df-9b5c-41cded99c229", status: 204, headers: {"x-request-id": "3e61d2b5-20c2-4954-8337-5cfe7a3dace5", "date": "Wed, 30 Apr 2025 06:43:36 GMT"} }) sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10352025-04-30T06:43:37.453ZDEBGtest_unauthorized (init_with_steps): Running step start_crucible_pantry
10362025-04-30T06:43:37.454ZDEBGtest_unauthorized (dropshot): registered endpoint kind = pantry local_addr = [::1]:41451 method = GET path = /crucible/pantry/0 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10372025-04-30T06:43:37.454ZDEBGtest_unauthorized (dropshot): registered endpoint kind = pantry local_addr = [::1]:41451 method = GET path = /crucible/pantry/0/job/{id}/is-finished sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10382025-04-30T06:43:37.454ZDEBGtest_unauthorized (dropshot): registered endpoint kind = pantry local_addr = [::1]:41451 method = GET path = /crucible/pantry/0/job/{id}/ok sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10392025-04-30T06:43:37.454ZDEBGtest_unauthorized (dropshot): registered endpoint kind = pantry local_addr = [::1]:41451 method = DELETE path = /crucible/pantry/0/volume/{id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10402025-04-30T06:43:37.455ZDEBGtest_unauthorized (dropshot): registered endpoint kind = pantry local_addr = [::1]:41451 method = GET path = /crucible/pantry/0/volume/{id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10412025-04-30T06:43:37.455ZDEBGtest_unauthorized (dropshot): registered endpoint kind = pantry local_addr = [::1]:41451 method = POST path = /crucible/pantry/0/volume/{id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10422025-04-30T06:43:37.456ZDEBGtest_unauthorized (dropshot): registered endpoint kind = pantry local_addr = [::1]:41451 method = POST path = /crucible/pantry/0/volume/{id}/background sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10432025-04-30T06:43:37.456ZDEBGtest_unauthorized (dropshot): registered endpoint kind = pantry local_addr = [::1]:41451 method = POST path = /crucible/pantry/0/volume/{id}/bulk-write sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10442025-04-30T06:43:37.456ZDEBGtest_unauthorized (dropshot): registered endpoint kind = pantry local_addr = [::1]:41451 method = POST path = /crucible/pantry/0/volume/{id}/import-from-url sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10452025-04-30T06:43:37.456ZDEBGtest_unauthorized (dropshot): registered endpoint kind = pantry local_addr = [::1]:41451 method = POST path = /crucible/pantry/0/volume/{id}/scrub sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10462025-04-30T06:43:37.456ZDEBGtest_unauthorized (dropshot): registered endpoint kind = pantry local_addr = [::1]:41451 method = POST path = /crucible/pantry/0/volume/{id}/snapshot sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10472025-04-30T06:43:37.456ZINFOtest_unauthorized (dropshot): listening kind = pantry local_addr = [::1]:41451 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10482025-04-30T06:43:37.456ZDEBGtest_unauthorized (dropshot): successfully registered DTrace USDT probes kind = pantry local_addr = [::1]:41451 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10492025-04-30T06:43:37.457ZINFOtest_unauthorized (omicron_sled_agent::sim::Server): Started Simulated Crucible Pantry address = [::1]:41451 kind = pantry sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10502025-04-30T06:43:37.457ZDEBGtest_unauthorized (init_with_steps): Running step populate_internal_dns
10512025-04-30T06:43:37.457ZDEBGtest_unauthorized: Populating Internal DNS
10522025-04-30T06:43:37.605ZINFOtest_unauthorized: DNS population: DnsConfigParams { generation: Generation( 1, ), time_created: 2025-04-30T06:43:37.604573949Z, zones: [ DnsConfigZone { zone_name: "control-plane.oxide.internal", records: { "_repo-depot._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 48692, target: "b6d65341-167c-41df-9b5c-41cded99c229.sled.control-plane.oxide.internal", }, ), ], "_crucible-pantry._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 41451, target: "fad54123-a622-440a-b83e-b1328fb8bb68.host.control-plane.oxide.internal", }, ), ], "_nameservice._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 39462, target: "f74fcbb5-b3b5-4928-b67d-87dd9b2f844a.host.control-plane.oxide.internal", }, ), ], "_nexus._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 34029, target: "8d752adb-08b9-405f-a11b-66019bf447b5.host.control-plane.oxide.internal", }, ), ], "f74fcbb5-b3b5-4928-b67d-87dd9b2f844a.host": [ Aaaa( ::1, ), ], "8d752adb-08b9-405f-a11b-66019bf447b5.host": [ Aaaa( ::ffff:127.0.0.1, ), ], "b6d65341-167c-41df-9b5c-41cded99c229.sled": [ Aaaa( ::1, ), ], "_clickhouse._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 50203, target: "25ec4a94-4291-467e-94d2-7b939aabac46.host.control-plane.oxide.internal", }, ), ], "_oximeter-reader._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 9000, target: "25ec4a94-4291-467e-94d2-7b939aabac46.host.control-plane.oxide.internal", }, ), ], "_clickhouse-native._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 9000, target: "25ec4a94-4291-467e-94d2-7b939aabac46.host.control-plane.oxide.internal", }, ), ], "_dendrite._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 44264, target: "dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal", }, ), ], "1f73cf6e-41b0-491d-b97a-75b677d5c06f.host": [ Aaaa( ::1, ), ], "9a892c5e-6788-4649-8d0f-08ee55eaa359.host": [ Aaaa( ::1, ), ], "fad54123-a622-440a-b83e-b1328fb8bb68.host": [ Aaaa( ::1, ), ], "_mgs._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 43500, target: "dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal", }, ), ], "_clickhouse-admin-single-server._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 8888, target: "25ec4a94-4291-467e-94d2-7b939aabac46.host.control-plane.oxide.internal", }, ), ], "_external-dns._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 61288, target: "9a892c5e-6788-4649-8d0f-08ee55eaa359.host.control-plane.oxide.internal", }, ), ], "_mgd._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 33198, target: "dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal", }, ), ], "dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host": [ Aaaa( ::1, ), ], "_cockroach._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 46222, target: "1f73cf6e-41b0-491d-b97a-75b677d5c06f.host.control-plane.oxide.internal", }, ), ], "25ec4a94-4291-467e-94d2-7b939aabac46.host": [ Aaaa( ::1, ), ], }, }, ], }
10532025-04-30T06:43:37.624ZDEBGtest_unauthorized: client request body = Some(Body) method = PUT uri = http://[::1]:39462/config
10542025-04-30T06:43:37.637ZTRACtest_unauthorized (BackgroundTasks): reprocessing 0 outdated webhook globs... background_task = webhook_dispatcher
10552025-04-30T06:43:37.637ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_dispatcher resource = Database
10562025-04-30T06:43:37.638ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = webhook_dispatcher roles = RoleSet { roles: {} }
10572025-04-30T06:43:37.639ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_dispatcher resource = Database result = Ok(())
10582025-04-30T06:43:37.639ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
10592025-04-30T06:43:37.639ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
10602025-04-30T06:43:37.639ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
10612025-04-30T06:43:37.639ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
10622025-04-30T06:43:37.640ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
10632025-04-30T06:43:37.641ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 1.280116028s iteration = 1 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
10642025-04-30T06:43:37.641ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = region_replacement elapsed = 1.28164771s iteration = 1 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
10652025-04-30T06:43:37.641ZINFOtest_unauthorized (BackgroundTasks): all instance checks complete background_task = instance_watcher pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
10662025-04-30T06:43:37.642ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 1.280684841s iteration = 1 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
10672025-04-30T06:43:37.642ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database
10682025-04-30T06:43:37.642ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
10692025-04-30T06:43:37.643ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
10702025-04-30T06:43:37.643ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 1.274358095s iteration = 1 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
10712025-04-30T06:43:37.677ZINFOtest_unauthorized (http): accepted connection kind = dns local_addr = [::1]:39462 remote_addr = [::1]:57125
10722025-04-30T06:43:37.679ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10732025-04-30T06:43:37.683ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
10742025-04-30T06:43:37.683ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database
10752025-04-30T06:43:37.683ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
10762025-04-30T06:43:37.684ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
10772025-04-30T06:43:37.686ZTRACtest_unauthorized (http): incoming request kind = dns local_addr = [::1]:39462 method = PUT remote_addr = [::1]:57125 req_id = 36085e0c-427e-4300-95fd-ab5cd3783460 uri = /config
10782025-04-30T06:43:37.686ZINFOtest_unauthorized (store): attempting generation update new_generation = 1 req_id = 36085e0c-427e-4300-95fd-ab5cd3783460
10792025-04-30T06:43:37.686ZINFOtest_unauthorized (store): pruning trees for generations newer than 0
10802025-04-30T06:43:37.687ZDEBGtest_unauthorized (store): creating tree tree_name = generation_1_zone_control-plane.oxide.internal
10812025-04-30T06:43:37.692ZDEBGtest_unauthorized (store): updating current config
10822025-04-30T06:43:37.692ZDEBGtest_unauthorized (store): flushing default tree
10832025-04-30T06:43:37.693ZINFOtest_unauthorized (store): pruning trees for generations older than 1 keep = 3
10842025-04-30T06:43:37.693ZINFOtest_unauthorized (store): updated generation new_generation = 1 req_id = 36085e0c-427e-4300-95fd-ab5cd3783460
10852025-04-30T06:43:37.693ZINFOtest_unauthorized (http): request completed kind = dns latency_us = 7667 local_addr = [::1]:39462 method = PUT remote_addr = [::1]:57125 req_id = 36085e0c-427e-4300-95fd-ab5cd3783460 response_code = 204 uri = /config
10862025-04-30T06:43:37.709ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10872025-04-30T06:43:37.739ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
10882025-04-30T06:43:37.740ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
10892025-04-30T06:43:37.740ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
10902025-04-30T06:43:37.741ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
10912025-04-30T06:43:37.743ZDEBGtest_unauthorized: client response result = Ok(Response { url: "http://[::1]:39462/config", status: 204, headers: {"x-request-id": "36085e0c-427e-4300-95fd-ab5cd3783460", "date": "Wed, 30 Apr 2025 06:43:37 GMT"} })
10922025-04-30T06:43:37.752ZDEBGtest_unauthorized (init_with_steps): Running step configure_sled_agents
10932025-04-30T06:43:37.877ZDEBGtest_unauthorized: client request body = Some(Body) method = PUT uri = http://[::1]:57786/omicron-config
10942025-04-30T06:43:37.882ZTRACtest_unauthorized (BackgroundTasks): no more instances in need of reincarnation background_task = instance_reincarnation reincarnation_reason = start saga failed total_found = 0
10952025-04-30T06:43:37.885ZINFOtest_unauthorized (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
10962025-04-30T06:43:37.886ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 1.522534691s iteration = 1 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
10972025-04-30T06:43:37.889ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:35750 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10982025-04-30T06:43:37.894ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = PUT remote_addr = [::1]:35750 req_id = 733f65dd-bcb8-447d-ba13-ed4b1b558b9d sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /omicron-config
10992025-04-30T06:43:37.918ZINFOtest_unauthorized (dropshot): request completed latency_us = 24331 local_addr = [::1]:57786 method = PUT remote_addr = [::1]:35750 req_id = 733f65dd-bcb8-447d-ba13-ed4b1b558b9d response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /omicron-config
11002025-04-30T06:43:37.922ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
11012025-04-30T06:43:37.949ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
11022025-04-30T06:43:37.949ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
11032025-04-30T06:43:37.950ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
11042025-04-30T06:43:37.950ZTRACtest_unauthorized (BackgroundTasks): webhook dispatching completed successfully background_task = webhook_dispatcher events_dispatched = 0 events_without_receivers = 0
11052025-04-30T06:43:37.950ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = webhook_dispatcher elapsed = 1.578151904s iteration = 1 status = {"dispatched":[],"errors":[],"glob_version":"140.0.0","globs_reprocessed":{},"no_receivers":[]}
11062025-04-30T06:43:37.950ZWARNtest_unauthorized (BackgroundTasks): failed to read Silo/DNS/TLS configuration background_task = external_endpoints error = Internal Error: expected at least one external DNS zone
11072025-04-30T06:43:37.950ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 1.597674532s iteration = 1 status = {"error":"failed to read Silo/DNS/TLS configuration: Internal Error: expected at least one external DNS zone"}
11082025-04-30T06:43:37.994ZDEBGtest_unauthorized: client response result = Ok(Response { url: "http://[::1]:57786/omicron-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "733f65dd-bcb8-447d-ba13-ed4b1b558b9d", "content-length": "26", "date": "Wed, 30 Apr 2025 06:43:37 GMT"} })
11092025-04-30T06:43:37.995ZDEBGtest_unauthorized: client request body = Some(Body) method = PUT uri = http://[::1]:57786/network-bootstore-config
11102025-04-30T06:43:37.998ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = PUT remote_addr = [::1]:35750 req_id = f6128d11-f673-44df-971a-0a3194dbe1f6 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /network-bootstore-config
11112025-04-30T06:43:38.004ZINFOtest_unauthorized (dropshot): request completed latency_us = 2956 local_addr = [::1]:57786 method = PUT remote_addr = [::1]:35750 req_id = f6128d11-f673-44df-971a-0a3194dbe1f6 response_code = 204 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /network-bootstore-config
11122025-04-30T06:43:38.005ZDEBGtest_unauthorized: client response result = Ok(Response { url: "http://[::1]:57786/network-bootstore-config", status: 204, headers: {"x-request-id": "f6128d11-f673-44df-971a-0a3194dbe1f6", "date": "Wed, 30 Apr 2025 06:43:37 GMT"} })
11132025-04-30T06:43:38.048ZDEBGtest_unauthorized (init_with_steps): Running step start_nexus_external
11142025-04-30T06:43:38.048ZDEBGtest_unauthorized: Starting Nexus (external API)
11152025-04-30T06:43:41.434ZTRACtest_unauthorized (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
11162025-04-30T06:43:41.434ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
11172025-04-30T06:43:41.434ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
11182025-04-30T06:43:41.434ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
11192025-04-30T06:43:41.435ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
11202025-04-30T06:43:41.503ZINFOtest_unauthorized (BackgroundTasks): VPC route manager sled b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager
11212025-04-30T06:43:41.504ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager body = None method = GET uri = http://[::1]:57786/vpc-routes
11222025-04-30T06:43:41.505ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet
11232025-04-30T06:43:41.505ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = v2p_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
11242025-04-30T06:43:41.505ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
11252025-04-30T06:43:41.505ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
11262025-04-30T06:43:41.506ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
11272025-04-30T06:43:41.508ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
11282025-04-30T06:43:41.508ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
11292025-04-30T06:43:41.508ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
11302025-04-30T06:43:41.508ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
11312025-04-30T06:43:41.623ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet
11322025-04-30T06:43:41.624ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = tuf_artifact_replication resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
11332025-04-30T06:43:41.624ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database
11342025-04-30T06:43:41.624ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
11352025-04-30T06:43:41.628ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
11362025-04-30T06:43:41.629ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 5.259165204s iteration = 1 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
11372025-04-30T06:43:41.631ZINFOtest_unauthorized (BackgroundTasks): No bundles to collect background_task = support_bundle_collector
11382025-04-30T06:43:41.631ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 5.275126526s iteration = 1 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
11392025-04-30T06:43:41.632ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = []
11402025-04-30T06:43:41.632ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
11412025-04-30T06:43:41.634ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:38579 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
11422025-04-30T06:43:41.638ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:38579 req_id = 1dda6c51-075a-4369-a647-78bbcf9f42fb sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc-routes
11432025-04-30T06:43:41.643ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: e3 6a 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .j..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:53015 req_id = 758c5869-bdbb-431c-811d-0da6dce76e28
11442025-04-30T06:43:41.643ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 58218,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:53015 req_id = 758c5869-bdbb-431c-811d-0da6dce76e28
11452025-04-30T06:43:41.643ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
11462025-04-30T06:43:41.644ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
11472025-04-30T06:43:41.645ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:53015 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 758c5869-bdbb-431c-811d-0da6dce76e28
11482025-04-30T06:43:41.646ZINFOtest_unauthorized (dropshot): request completed latency_us = 7854 local_addr = [::1]:57786 method = GET remote_addr = [::1]:38579 req_id = 1dda6c51-075a-4369-a647-78bbcf9f42fb response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc-routes
11492025-04-30T06:43:41.648ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4286, tv_nsec: 447726404 } })
11502025-04-30T06:43:41.648ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
11512025-04-30T06:43:41.650ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager result = Ok(Response { url: "http://[::1]:57786/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "1dda6c51-075a-4369-a647-78bbcf9f42fb", "content-length": "2", "date": "Wed, 30 Apr 2025 06:43:41 GMT"} })
11522025-04-30T06:43:41.650ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
11532025-04-30T06:43:41.650ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11542025-04-30T06:43:41.651ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11552025-04-30T06:43:41.651ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11562025-04-30T06:43:41.655ZDEBGtest_unauthorized (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
11572025-04-30T06:43:41.655ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
11582025-04-30T06:43:41.655ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
11592025-04-30T06:43:41.656ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
11602025-04-30T06:43:41.661ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11612025-04-30T06:43:41.672ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
11622025-04-30T06:43:41.672ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet
11632025-04-30T06:43:41.672ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = v2p_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
11642025-04-30T06:43:41.672ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
11652025-04-30T06:43:41.672ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
11662025-04-30T06:43:41.673ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
11672025-04-30T06:43:41.675ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11682025-04-30T06:43:41.687ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
11692025-04-30T06:43:41.687ZTRACtest_unauthorized (BackgroundTasks): successful lookup of siloed resource "Vpc" using built-in user background_task = service_firewall_rule_propagation
11702025-04-30T06:43:41.687ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
11712025-04-30T06:43:41.687ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
11722025-04-30T06:43:41.687ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
11732025-04-30T06:43:41.687ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
11742025-04-30T06:43:41.688ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
11752025-04-30T06:43:41.689ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11762025-04-30T06:43:41.696ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
11772025-04-30T06:43:41.696ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database
11782025-04-30T06:43:41.696ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
11792025-04-30T06:43:41.697ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
11802025-04-30T06:43:41.700ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: cf 61 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .a..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:61677 req_id = 74f36a6e-0363-447e-be42-4a6970fb4487
11812025-04-30T06:43:41.700ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 53089,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:61677 req_id = 74f36a6e-0363-447e-be42-4a6970fb4487
11822025-04-30T06:43:41.700ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
11832025-04-30T06:43:41.700ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:61677 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 74f36a6e-0363-447e-be42-4a6970fb4487
11842025-04-30T06:43:41.777ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
11852025-04-30T06:43:41.788ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
11862025-04-30T06:43:41.788ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
11872025-04-30T06:43:41.788ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
11882025-04-30T06:43:41.790ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 7b 8a 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c {............_cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:57258 req_id = eea18918-107c-470b-992a-53ecbd5334b4
11892025-04-30T06:43:41.790ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 31626,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:57258 req_id = eea18918-107c-470b-992a-53ecbd5334b4
11902025-04-30T06:43:41.790ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
11912025-04-30T06:43:41.791ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 31626, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:57258 req_id = eea18918-107c-470b-992a-53ecbd5334b4
11922025-04-30T06:43:41.791ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11932025-04-30T06:43:41.795ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
11942025-04-30T06:43:41.795ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
11952025-04-30T06:43:41.795ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
11962025-04-30T06:43:41.796ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
11972025-04-30T06:43:41.797ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
11982025-04-30T06:43:41.797ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
11992025-04-30T06:43:41.797ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12002025-04-30T06:43:41.798ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12012025-04-30T06:43:41.802ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet
12022025-04-30T06:43:41.802ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = tuf_artifact_replication resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
12032025-04-30T06:43:41.802ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database
12042025-04-30T06:43:41.802ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
12052025-04-30T06:43:41.803ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
12062025-04-30T06:43:41.812ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
12072025-04-30T06:43:41.812ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
12082025-04-30T06:43:41.812ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
12092025-04-30T06:43:41.826ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
12102025-04-30T06:43:41.826ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
12112025-04-30T06:43:41.826ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12122025-04-30T06:43:41.827ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12132025-04-30T06:43:41.874ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = v2p_manager body = None method = GET uri = http://[::1]:57786/v2p
12142025-04-30T06:43:41.875ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12152025-04-30T06:43:41.880ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
12162025-04-30T06:43:41.880ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database
12172025-04-30T06:43:41.880ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
12182025-04-30T06:43:41.881ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
12192025-04-30T06:43:41.882ZINFOtest_unauthorized (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager sled = sim-b6d65341
12202025-04-30T06:43:41.882ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[::1]:57786/eip-gateways
12212025-04-30T06:43:41.882ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12222025-04-30T06:43:41.889ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
12232025-04-30T06:43:41.889ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
12242025-04-30T06:43:41.889ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12252025-04-30T06:43:41.890ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12262025-04-30T06:43:41.891ZINFOtest_unauthorized (InternalApi): set cockroachdb setting setting = cluster.preserve_downgrade_option value = 22.1
12272025-04-30T06:43:41.892ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
12282025-04-30T06:43:41.892ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
12292025-04-30T06:43:41.892ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
12302025-04-30T06:43:41.893ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:49469 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
12312025-04-30T06:43:41.893ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = PUT remote_addr = [::1]:38579 req_id = 2a4fb789-9afd-4e76-a6ca-3639cec0e4c0 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /eip-gateways
12322025-04-30T06:43:41.895ZINFOtest_unauthorized (dropshot): request completed latency_us = 2244 local_addr = [::1]:57786 method = PUT remote_addr = [::1]:38579 req_id = 2a4fb789-9afd-4e76-a6ca-3639cec0e4c0 response_code = 204 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /eip-gateways
12332025-04-30T06:43:41.897ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:49469 req_id = bcb85574-e4a8-40ee-a394-3d40413d0350 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /v2p
12342025-04-30T06:43:41.898ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager result = Ok(Response { url: "http://[::1]:57786/eip-gateways", status: 204, headers: {"x-request-id": "2a4fb789-9afd-4e76-a6ca-3639cec0e4c0", "date": "Wed, 30 Apr 2025 06:43:41 GMT"} })
12352025-04-30T06:43:41.899ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 5.533977239s iteration = 1 status = {}
12362025-04-30T06:43:41.900ZINFOtest_unauthorized (dropshot): request completed latency_us = 2508 local_addr = [::1]:57786 method = GET remote_addr = [::1]:49469 req_id = bcb85574-e4a8-40ee-a394-3d40413d0350 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /v2p
12372025-04-30T06:43:41.900ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = tuf_artifact_replication body = Some(Body) method = PUT uri = http://[::1]:57786/artifacts-config
12382025-04-30T06:43:41.902ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
12392025-04-30T06:43:41.902ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12402025-04-30T06:43:41.903ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12412025-04-30T06:43:41.904ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:57433 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
12422025-04-30T06:43:41.907ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "3fea4b1e-c772-4b1d-8b3a-e62605d6b2c0", "content-length": "835", "date": "Wed, 30 Apr 2025 06:43:41 GMT"} })
12432025-04-30T06:43:41.909ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
12442025-04-30T06:43:41.909ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = v2p_manager result = Ok(Response { url: "http://[::1]:57786/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "bcb85574-e4a8-40ee-a394-3d40413d0350", "content-length": "2", "date": "Wed, 30 Apr 2025 06:43:41 GMT"} })
12452025-04-30T06:43:41.909ZINFOtest_unauthorized (BackgroundTasks): found opte v2p mappings background_task = v2p_manager interfaces = {} sled = sim-b6d65341
12462025-04-30T06:43:41.909ZINFOtest_unauthorized (BackgroundTasks): v2p mappings to delete background_task = v2p_manager mappings = [] sled = sim-b6d65341
12472025-04-30T06:43:41.910ZINFOtest_unauthorized (BackgroundTasks): v2p mappings to add background_task = v2p_manager mappings = [] sled = sim-b6d65341
12482025-04-30T06:43:41.913ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 5.554482444s iteration = 1 status = {}
12492025-04-30T06:43:41.915ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Rack { parent: Fleet, key: c19a698f-c6f9-4a17-ae30-20d711b8f7dc, lookup_type: ById(c19a698f-c6f9-4a17-ae30-20d711b8f7dc) }
12502025-04-30T06:43:41.915ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
12512025-04-30T06:43:41.915ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
12522025-04-30T06:43:41.915ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
12532025-04-30T06:43:41.916ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
12542025-04-30T06:43:41.920ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = PUT remote_addr = [::1]:57433 req_id = 5afec977-1368-46b5-91ab-0199e6f0f44a sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /artifacts-config
12552025-04-30T06:43:41.920ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
12562025-04-30T06:43:41.921ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12572025-04-30T06:43:41.922ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12582025-04-30T06:43:41.925ZINFOtest_unauthorized (ArtifactStore): No ledger in /var/tmp/omicron_tmp/.tmprM0M8I/artifact-config.json server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
12592025-04-30T06:43:41.925ZDEBGtest_unauthorized (ArtifactStore): Failed to read ledger: Not found in storage path = /var/tmp/omicron_tmp/.tmprM0M8I/artifact-config.json server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
12602025-04-30T06:43:41.925ZINFOtest_unauthorized (ArtifactStore): No ledger in /var/tmp/omicron_tmp/.tmpzboduk/artifact-config.json server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
12612025-04-30T06:43:41.925ZDEBGtest_unauthorized (ArtifactStore): Failed to read ledger: Not found in storage path = /var/tmp/omicron_tmp/.tmpzboduk/artifact-config.json server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
12622025-04-30T06:43:41.925ZINFOtest_unauthorized (ArtifactStore): Writing ledger to /var/tmp/omicron_tmp/.tmprM0M8I/.artifact-config.json.tmp server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
12632025-04-30T06:43:41.926ZINFOtest_unauthorized (ArtifactStore): Writing ledger to /var/tmp/omicron_tmp/.tmpzboduk/.artifact-config.json.tmp server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
12642025-04-30T06:43:41.930ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
12652025-04-30T06:43:41.931ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12662025-04-30T06:43:41.933ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12672025-04-30T06:43:41.934ZINFOtest_unauthorized (ArtifactStore): Starting delete reconciler generation = 1 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
12682025-04-30T06:43:41.934ZINFOtest_unauthorized (dropshot): request completed latency_us = 14167 local_addr = [::1]:57786 method = PUT remote_addr = [::1]:57433 req_id = 5afec977-1368-46b5-91ab-0199e6f0f44a response_code = 204 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /artifacts-config
12692025-04-30T06:43:41.935ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12702025-04-30T06:43:41.942ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Rack { parent: Fleet, key: c19a698f-c6f9-4a17-ae30-20d711b8f7dc, lookup_type: ById(c19a698f-c6f9-4a17-ae30-20d711b8f7dc) } result = Ok(())
12712025-04-30T06:43:41.942ZDEBGtest_unauthorized (InternalApi): updating rack subnet for rack c19a698f-c6f9-4a17-ae30-20d711b8f7dc to Some( V6( Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56, }, ), )
12722025-04-30T06:43:41.942ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
12732025-04-30T06:43:41.942ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
12742025-04-30T06:43:41.943ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
12752025-04-30T06:43:41.945ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = tuf_artifact_replication result = Ok(Response { url: "http://[::1]:57786/artifacts-config", status: 204, headers: {"x-request-id": "5afec977-1368-46b5-91ab-0199e6f0f44a", "date": "Wed, 30 Apr 2025 06:43:41 GMT"} })
12762025-04-30T06:43:41.945ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = tuf_artifact_replication body = None method = GET uri = http://[::1]:57786/artifacts
12772025-04-30T06:43:41.945ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
12782025-04-30T06:43:41.946ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
12792025-04-30T06:43:41.946ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
12802025-04-30T06:43:41.946ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12812025-04-30T06:43:41.946ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12822025-04-30T06:43:41.947ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:57433 req_id = 67c688aa-508a-4ad9-87e3-e375ea511cad sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /artifacts
12832025-04-30T06:43:41.955ZINFOtest_unauthorized (dropshot): request completed latency_us = 7790 local_addr = [::1]:57786 method = GET remote_addr = [::1]:57433 req_id = 67c688aa-508a-4ad9-87e3-e375ea511cad response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /artifacts
12842025-04-30T06:43:41.955ZINFOtest_unauthorized (InternalApi): Using static configuration for external switchports
12852025-04-30T06:43:41.956ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
12862025-04-30T06:43:41.956ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
12872025-04-30T06:43:41.956ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
12882025-04-30T06:43:41.957ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
12892025-04-30T06:43:41.957ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
12902025-04-30T06:43:41.957ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12912025-04-30T06:43:41.957ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12922025-04-30T06:43:41.958ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = tuf_artifact_replication result = Ok(Response { url: "http://[::1]:57786/artifacts", status: 200, headers: {"content-type": "application/json", "x-request-id": "67c688aa-508a-4ad9-87e3-e375ea511cad", "content-length": "26", "date": "Wed, 30 Apr 2025 06:43:41 GMT"} })
12932025-04-30T06:43:41.958ZINFOtest_unauthorized (BackgroundTasks): Successfully got artifact list background_task = tuf_artifact_replication sled = http://[::1]:57786
12942025-04-30T06:43:41.960ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = tuf_artifact_replication elapsed = 5.589615875s iteration = 1 status = {"generation":1,"last_run_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":1,"put_config_err":0,"put_config_ok":1,"put_err":0,"put_ok":0},"lifetime_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":1,"put_config_err":0,"put_config_ok":1,"put_err":0,"put_ok":0},"local_repos":0,"request_debug_ringbuf":[{"generation":1,"operation":"put_config","target_sled":"b6d65341-167c-41df-9b5c-41cded99c229","time":"2025-04-30T06:43:41.945134122Z"},{"operation":"list","target_sled":"b6d65341-167c-41df-9b5c-41cded99c229","time":"2025-04-30T06:43:41.958454211Z"}]}
12952025-04-30T06:43:41.966ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
12962025-04-30T06:43:41.966ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
12972025-04-30T06:43:41.966ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12982025-04-30T06:43:41.966ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12992025-04-30T06:43:41.972ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13002025-04-30T06:43:41.979ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
13012025-04-30T06:43:41.979ZTRACtest_unauthorized (BackgroundTasks): successful lookup of siloed resource "Vpc" using built-in user background_task = service_firewall_rule_propagation
13022025-04-30T06:43:41.979ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13032025-04-30T06:43:41.979ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13042025-04-30T06:43:41.979ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13052025-04-30T06:43:41.990ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a1a96674-c10c-476c-90e2-5d492a086fb1", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:41 GMT"} })
13062025-04-30T06:43:41.990ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
13072025-04-30T06:43:41.995ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13082025-04-30T06:43:41.995ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13092025-04-30T06:43:41.995ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13102025-04-30T06:43:42.005ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13112025-04-30T06:43:42.005ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13122025-04-30T06:43:42.007ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13132025-04-30T06:43:42.010ZINFOtest_unauthorized (InternalApi): Recording Rack Network Configuration
13142025-04-30T06:43:42.010ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
13152025-04-30T06:43:42.010ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
13162025-04-30T06:43:42.011ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
13172025-04-30T06:43:42.013ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13182025-04-30T06:43:42.013ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13192025-04-30T06:43:42.013ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13202025-04-30T06:43:42.022ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") }
13212025-04-30T06:43:42.022ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
13222025-04-30T06:43:42.022ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13232025-04-30T06:43:42.022ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13242025-04-30T06:43:42.023ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13252025-04-30T06:43:42.033ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
13262025-04-30T06:43:42.033ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13272025-04-30T06:43:42.033ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13282025-04-30T06:43:42.034ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13292025-04-30T06:43:42.044ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
13302025-04-30T06:43:42.044ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13312025-04-30T06:43:42.044ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13322025-04-30T06:43:42.045ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13332025-04-30T06:43:42.047ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13342025-04-30T06:43:42.073ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
13352025-04-30T06:43:42.073ZTRACtest_unauthorized (BackgroundTasks): successful lookup of siloed resource "VpcSubnet" using built-in user background_task = service_firewall_rule_propagation
13362025-04-30T06:43:42.074ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") }
13372025-04-30T06:43:42.085ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
13382025-04-30T06:43:42.098ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13392025-04-30T06:43:42.098ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13402025-04-30T06:43:42.099ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13412025-04-30T06:43:42.103ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
13422025-04-30T06:43:42.103ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13432025-04-30T06:43:42.103ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13442025-04-30T06:43:42.104ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13452025-04-30T06:43:42.108ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
13462025-04-30T06:43:42.109ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13472025-04-30T06:43:42.109ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13482025-04-30T06:43:42.109ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13492025-04-30T06:43:42.120ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13502025-04-30T06:43:42.132ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
13512025-04-30T06:43:42.132ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13522025-04-30T06:43:42.133ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13532025-04-30T06:43:42.133ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13542025-04-30T06:43:42.134ZTRACtest_unauthorized (InternalApi): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
13552025-04-30T06:43:42.134ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
13562025-04-30T06:43:42.135ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
13572025-04-30T06:43:42.135ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
13582025-04-30T06:43:42.135ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
13592025-04-30T06:43:42.139ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a09aa00e-1b9d-4a9f-90b2-803f85ad5bb4", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:41 GMT"} })
13602025-04-30T06:43:42.140ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
13612025-04-30T06:43:42.172ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13622025-04-30T06:43:42.172ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13632025-04-30T06:43:42.214ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13642025-04-30T06:43:42.214ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13652025-04-30T06:43:42.217ZDEBGtest_unauthorized (InternalApi): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
13662025-04-30T06:43:42.217ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
13672025-04-30T06:43:42.217ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
13682025-04-30T06:43:42.218ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
13692025-04-30T06:43:42.222ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13702025-04-30T06:43:42.222ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13712025-04-30T06:43:42.223ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13722025-04-30T06:43:42.228ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = IpPool { parent: Fleet, key: fd3fbfbd-d99f-4924-a84f-de372a914746, lookup_type: ByName("oxide-service-pool") }
13732025-04-30T06:43:42.228ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
13742025-04-30T06:43:42.228ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
13752025-04-30T06:43:42.228ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
13762025-04-30T06:43:42.230ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
13772025-04-30T06:43:42.236ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13782025-04-30T06:43:42.236ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13792025-04-30T06:43:42.236ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13802025-04-30T06:43:42.236ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13812025-04-30T06:43:42.242ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = IpPool { parent: Fleet, key: fd3fbfbd-d99f-4924-a84f-de372a914746, lookup_type: ByName("oxide-service-pool") } result = Ok(())
13822025-04-30T06:43:42.242ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
13832025-04-30T06:43:42.242ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
13842025-04-30T06:43:42.242ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
13852025-04-30T06:43:42.254ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13862025-04-30T06:43:42.254ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13872025-04-30T06:43:42.273ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13882025-04-30T06:43:42.283ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") }
13892025-04-30T06:43:42.283ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
13902025-04-30T06:43:42.283ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13912025-04-30T06:43:42.283ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13922025-04-30T06:43:42.284ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13932025-04-30T06:43:42.284ZTRACtest_unauthorized (InternalApi): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = IpPool { parent: Fleet, key: fd3fbfbd-d99f-4924-a84f-de372a914746, lookup_type: ByName("oxide-service-pool") }
13942025-04-30T06:43:42.284ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
13952025-04-30T06:43:42.284ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
13962025-04-30T06:43:42.285ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
13972025-04-30T06:43:42.285ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
13982025-04-30T06:43:42.290ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
13992025-04-30T06:43:42.290ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14002025-04-30T06:43:42.290ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14012025-04-30T06:43:42.291ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14022025-04-30T06:43:42.296ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "178ebf81-e2f8-487b-ab52-9a8b27d82ee8", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:41 GMT"} })
14032025-04-30T06:43:42.296ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
14042025-04-30T06:43:42.301ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14052025-04-30T06:43:42.305ZDEBGtest_unauthorized (InternalApi): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = IpPool { parent: Fleet, key: fd3fbfbd-d99f-4924-a84f-de372a914746, lookup_type: ByName("oxide-service-pool") } result = Ok(())
14062025-04-30T06:43:42.305ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
14072025-04-30T06:43:42.306ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14082025-04-30T06:43:42.306ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14092025-04-30T06:43:42.306ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14102025-04-30T06:43:42.312ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14112025-04-30T06:43:42.321ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
14122025-04-30T06:43:42.321ZTRACtest_unauthorized (BackgroundTasks): successful lookup of siloed resource "VpcSubnet" using built-in user background_task = service_firewall_rule_propagation
14132025-04-30T06:43:42.321ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") }
14142025-04-30T06:43:42.321ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
14152025-04-30T06:43:42.321ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14162025-04-30T06:43:42.321ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14172025-04-30T06:43:42.322ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14182025-04-30T06:43:42.323ZTRACtest_unauthorized (InternalApi): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = IpPool { parent: Fleet, key: fd3fbfbd-d99f-4924-a84f-de372a914746, lookup_type: ByName("oxide-service-pool") }
14192025-04-30T06:43:42.323ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
14202025-04-30T06:43:42.323ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
14212025-04-30T06:43:42.323ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
14222025-04-30T06:43:42.324ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
14232025-04-30T06:43:42.338ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
14242025-04-30T06:43:42.338ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14252025-04-30T06:43:42.338ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14262025-04-30T06:43:42.340ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14272025-04-30T06:43:42.349ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14282025-04-30T06:43:42.352ZDEBGtest_unauthorized (InternalApi): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = IpPool { parent: Fleet, key: fd3fbfbd-d99f-4924-a84f-de372a914746, lookup_type: ByName("oxide-service-pool") } result = Ok(())
14292025-04-30T06:43:42.355ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
14302025-04-30T06:43:42.355ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14312025-04-30T06:43:42.355ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14322025-04-30T06:43:42.356ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14332025-04-30T06:43:42.371ZTRACtest_unauthorized (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = BlueprintConfig
14342025-04-30T06:43:42.371ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
14352025-04-30T06:43:42.371ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
14362025-04-30T06:43:42.371ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
14372025-04-30T06:43:42.372ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
14382025-04-30T06:43:42.374ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14392025-04-30T06:43:42.381ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
14402025-04-30T06:43:42.382ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14412025-04-30T06:43:42.382ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14422025-04-30T06:43:42.382ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14432025-04-30T06:43:42.385ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14442025-04-30T06:43:42.391ZDEBGtest_unauthorized (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = BlueprintConfig result = Ok(())
14452025-04-30T06:43:42.473ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "adadb252-77c5-4059-be11-c398c1fd6498", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:41 GMT"} })
14462025-04-30T06:43:42.473ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
14472025-04-30T06:43:42.473ZDEBGtest_unauthorized (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
14482025-04-30T06:43:42.474ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet
14492025-04-30T06:43:42.474ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
14502025-04-30T06:43:42.474ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14512025-04-30T06:43:42.474ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14522025-04-30T06:43:42.474ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14532025-04-30T06:43:42.479ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14542025-04-30T06:43:42.490ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
14552025-04-30T06:43:42.520ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14562025-04-30T06:43:42.520ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14572025-04-30T06:43:42.522ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14582025-04-30T06:43:42.533ZTRACtest_unauthorized (service-firewall-rule-progator): fetched allowlist from DB allowed = AllowList { id: 001de000-a110-4000-8000-000000000000, time_created: 2025-04-30T06:06:08.248500Z, time_modified: 2025-04-30T06:06:08.248500Z, allowed_ips: None } background_task = service_firewall_rule_propagation
14592025-04-30T06:43:42.533ZDEBGtest_unauthorized (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
14602025-04-30T06:43:42.533ZDEBGtest_unauthorized (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = []
14612025-04-30T06:43:42.533ZDEBGtest_unauthorized (service-firewall-rule-progator): resolved 0 rules for sleds background_task = service_firewall_rule_propagation
14622025-04-30T06:43:42.636ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ab310426-0df9-4c7b-b77b-48f5dd7c3bb9", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:41 GMT"} })
14632025-04-30T06:43:42.636ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
14642025-04-30T06:43:42.702ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d7fc246b-2655-4c41-9678-597dd02a9fb1", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:42 GMT"} })
14652025-04-30T06:43:42.744ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
14662025-04-30T06:43:42.751ZINFOtest_unauthorized (InternalApi): inserted blueprint blueprint_id = edf03f2e-45f1-4a87-afc2-cc34e9686976
14672025-04-30T06:43:42.752ZTRACtest_unauthorized (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = BlueprintConfig
14682025-04-30T06:43:42.752ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
14692025-04-30T06:43:42.782ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
14702025-04-30T06:43:42.782ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
14712025-04-30T06:43:42.783ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
14722025-04-30T06:43:42.788ZDEBGtest_unauthorized (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = []
14732025-04-30T06:43:42.788ZDEBGtest_unauthorized (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
14742025-04-30T06:43:42.806ZINFOtest_unauthorized (service-firewall-rule-progator): updated firewall rules on 0 sleds background_task = service_firewall_rule_propagation
14752025-04-30T06:43:42.843ZDEBGtest_unauthorized (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 6.479034544s
14762025-04-30T06:43:42.843ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 6.479120375s iteration = 1 status = {}
14772025-04-30T06:43:42.846ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2221a17b-9316-4a8f-a5af-d3c6b04fdd70", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:42 GMT"} })
14782025-04-30T06:43:42.846ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
14792025-04-30T06:43:42.847ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14802025-04-30T06:43:42.853ZDEBGtest_unauthorized (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = BlueprintConfig result = Ok(())
14812025-04-30T06:43:42.855ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1c2d0620-9c56-4519-941f-e8102d5d3fd7", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:42 GMT"} })
14822025-04-30T06:43:42.855ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
14832025-04-30T06:43:42.856ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fb349157-f015-4a98-81d8-6835bd6a09f9", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:42 GMT"} })
14842025-04-30T06:43:42.856ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
14852025-04-30T06:43:42.857ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "758319a5-3520-4fb8-8a20-e5627d830055", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:42 GMT"} })
14862025-04-30T06:43:42.857ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
14872025-04-30T06:43:42.858ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d205b88e-fdbb-421d-803e-591fe135f0fb", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:42 GMT"} })
14882025-04-30T06:43:42.858ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
14892025-04-30T06:43:42.859ZINFOtest_unauthorized (InternalApi): No networking records needed for cockroach_db service
14902025-04-30T06:43:42.859ZINFOtest_unauthorized (InternalApi): No networking records needed for clickhouse service
14912025-04-30T06:43:42.860ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5c7f0fc8-016c-4251-9f76-5d9c01a00aac", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:42 GMT"} })
14922025-04-30T06:43:42.860ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
14932025-04-30T06:43:42.896ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "40ae4de2-de6a-44ae-91ee-9b52eef22be3", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:42 GMT"} })
14942025-04-30T06:43:42.902ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
14952025-04-30T06:43:42.908ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f365504f-1b86-476d-813e-6d82c30a78ca", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:42 GMT"} })
14962025-04-30T06:43:42.909ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
14972025-04-30T06:43:42.910ZINFOtest_unauthorized (InternalApi): Inserted networking records for nexus service
14982025-04-30T06:43:42.915ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1d44684b-3145-4d2d-b5a8-b764e45c6665", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:42 GMT"} })
14992025-04-30T06:43:42.915ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
15002025-04-30T06:43:42.917ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "201c9a11-8543-4c40-9853-4b78f0e313de", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:42 GMT"} })
15012025-04-30T06:43:42.917ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
15022025-04-30T06:43:42.920ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "feb1e64b-b275-4e0b-a323-84e60e75f0f8", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:42 GMT"} })
15032025-04-30T06:43:42.920ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
15042025-04-30T06:43:42.922ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "67d12d5b-dbb5-484c-a49e-d72995e43167", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:42 GMT"} })
15052025-04-30T06:43:42.923ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
15062025-04-30T06:43:42.923ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9e1b240f-1bac-438b-9e3e-8393136ff109", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:42 GMT"} })
15072025-04-30T06:43:42.924ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
15082025-04-30T06:43:42.924ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d889f0d0-2aee-4fde-8f2b-bc203fb8e231", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:42 GMT"} })
15092025-04-30T06:43:42.925ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
15102025-04-30T06:43:42.926ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e9840795-95a0-46c5-8700-784c5ccdf203", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:42 GMT"} })
15112025-04-30T06:43:42.926ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
15122025-04-30T06:43:42.931ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a1aba3d8-ded8-4ef6-a4f1-106fb8106383", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:42 GMT"} })
15132025-04-30T06:43:42.931ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
15142025-04-30T06:43:42.933ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0e52719-4b3a-465c-980d-b8ba80a9075f", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:42 GMT"} })
15152025-04-30T06:43:42.933ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
15162025-04-30T06:43:42.933ZINFOtest_unauthorized (InternalApi): Inserted networking records for external_dns service
15172025-04-30T06:43:42.933ZINFOtest_unauthorized (InternalApi): No networking records needed for internal_dns service
15182025-04-30T06:43:42.933ZINFOtest_unauthorized (InternalApi): No networking records needed for crucible_pantry service
15192025-04-30T06:43:42.933ZINFOtest_unauthorized (InternalApi): Inserted service networking records
15202025-04-30T06:43:42.933ZINFOtest_unauthorized (InternalApi): Inserted physical disks
15212025-04-30T06:43:42.933ZINFOtest_unauthorized (InternalApi): Inserted zpools
15222025-04-30T06:43:42.933ZINFOtest_unauthorized (InternalApi): Inserted datasets
15232025-04-30T06:43:42.938ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "dd8d31ce-3b90-482d-b650-1ce42fdc71fd", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:42 GMT"} })
15242025-04-30T06:43:42.938ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
15252025-04-30T06:43:42.941ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2bbdf8e0-0dd0-4112-a8ca-48f17c4d0b16", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:42 GMT"} })
15262025-04-30T06:43:42.942ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
15272025-04-30T06:43:42.945ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "83b73916-a3fe-4443-84da-05b468e011b1", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:42 GMT"} })
15282025-04-30T06:43:42.945ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
15292025-04-30T06:43:42.946ZINFOtest_unauthorized (InternalApi): Populated DNS tables for internal DNS
15302025-04-30T06:43:42.947ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "90b54d33-0b77-48e6-a3cb-ebc0dd63bcf5", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:42 GMT"} })
15312025-04-30T06:43:42.948ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
15322025-04-30T06:43:42.950ZINFOtest_unauthorized (InternalApi): Populated DNS tables for external DNS
15332025-04-30T06:43:42.951ZTRACtest_unauthorized (InternalApi): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
15342025-04-30T06:43:42.951ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
15352025-04-30T06:43:42.951ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15362025-04-30T06:43:42.951ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
15372025-04-30T06:43:42.952ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15382025-04-30T06:43:42.952ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ee5310c4-0e39-434f-a52b-cb1a561103a5", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:42 GMT"} })
15392025-04-30T06:43:42.953ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
15402025-04-30T06:43:42.954ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15412025-04-30T06:43:42.957ZDEBGtest_unauthorized (InternalApi): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
15422025-04-30T06:43:42.957ZTRACtest_unauthorized (InternalApi): authorize begin action = ModifyPolicy actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
15432025-04-30T06:43:42.957ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
15442025-04-30T06:43:42.957ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15452025-04-30T06:43:42.957ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
15462025-04-30T06:43:42.957ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15472025-04-30T06:43:42.959ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ed1e8069-a526-49bc-9ab0-f262e65d0797", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:42 GMT"} })
15482025-04-30T06:43:42.959ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
15492025-04-30T06:43:42.960ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15502025-04-30T06:43:42.961ZDEBGtest_unauthorized (InternalApi): authorize result action = ModifyPolicy actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
15512025-04-30T06:43:42.962ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0a6c6c60-701d-4aca-bb81-1da6ea78a25d", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:42 GMT"} })
15522025-04-30T06:43:42.962ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
15532025-04-30T06:43:42.969ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "42abf8e7-361a-4f3d-9eb5-b764d73bdea6", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:42 GMT"} })
15542025-04-30T06:43:42.969ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
15552025-04-30T06:43:42.970ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0832450d-95bc-425e-94fd-f67a921a3b72", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:42 GMT"} })
15562025-04-30T06:43:42.970ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
15572025-04-30T06:43:42.974ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "dd3c6e93-328b-4157-8987-4a686582acf7", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:42 GMT"} })
15582025-04-30T06:43:42.974ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
15592025-04-30T06:43:42.975ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1341ad48-498a-4874-b40e-4f6df726d70a", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:42 GMT"} })
15602025-04-30T06:43:42.975ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
15612025-04-30T06:43:42.977ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c5bc7cc1-d894-4c78-ae99-5cdeec3c4fee", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:42 GMT"} })
15622025-04-30T06:43:42.978ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
15632025-04-30T06:43:42.978ZTRACtest_unauthorized (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = DnsConfig
15642025-04-30T06:43:42.978ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
15652025-04-30T06:43:42.978ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15662025-04-30T06:43:42.978ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
15672025-04-30T06:43:42.978ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15682025-04-30T06:43:42.980ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "15b654de-47eb-458f-886f-5092acefddae", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:42 GMT"} })
15692025-04-30T06:43:42.981ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
15702025-04-30T06:43:42.981ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15712025-04-30T06:43:42.987ZDEBGtest_unauthorized (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = DnsConfig result = Ok(())
15722025-04-30T06:43:42.987ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = DnsConfig
15732025-04-30T06:43:42.987ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
15742025-04-30T06:43:42.987ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15752025-04-30T06:43:42.987ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
15762025-04-30T06:43:42.987ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15772025-04-30T06:43:42.990ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15782025-04-30T06:43:42.996ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = DnsConfig result = Ok(())
15792025-04-30T06:43:43.003ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d9f6cc8f-0e39-4bdc-aa93-73cc0690906f", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:42 GMT"} })
15802025-04-30T06:43:43.003ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
15812025-04-30T06:43:43.006ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = DnsConfig
15822025-04-30T06:43:43.006ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
15832025-04-30T06:43:43.006ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15842025-04-30T06:43:43.006ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
15852025-04-30T06:43:43.007ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15862025-04-30T06:43:43.009ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5dbeef12-2323-42e6-bfec-7c3b4c5ea0d4", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:42 GMT"} })
15872025-04-30T06:43:43.010ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
15882025-04-30T06:43:43.010ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15892025-04-30T06:43:43.028ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = DnsConfig result = Ok(())
15902025-04-30T06:43:43.031ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f4bb5a70-d364-400e-8beb-590d09bb099f", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:42 GMT"} })
15912025-04-30T06:43:43.032ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
15922025-04-30T06:43:43.036ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b52a6334-e755-435a-ab13-48d79b7ac53f", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:42 GMT"} })
15932025-04-30T06:43:43.036ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
15942025-04-30T06:43:43.037ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ff8b4d2e-2257-4853-8d4c-1ed5d7016668", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:42 GMT"} })
15952025-04-30T06:43:43.038ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
15962025-04-30T06:43:43.039ZINFOtest_unauthorized (InternalApi): Created recovery silo
15972025-04-30T06:43:43.039ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cfa6b4d4-304d-40c0-aa59-67b92726557a", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:42 GMT"} })
15982025-04-30T06:43:43.040ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
15992025-04-30T06:43:43.040ZINFOtest_unauthorized (InternalApi): Created recovery user
16002025-04-30T06:43:43.041ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "de91135b-7163-4289-9c1e-e0493ac051a3", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:42 GMT"} })
16012025-04-30T06:43:43.042ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
16022025-04-30T06:43:43.044ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3a0faa80-8724-4566-8326-15991f5d2d07", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:42 GMT"} })
16032025-04-30T06:43:43.044ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
16042025-04-30T06:43:43.044ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16052025-04-30T06:43:43.044ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
16062025-04-30T06:43:43.044ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
16072025-04-30T06:43:43.045ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
16082025-04-30T06:43:43.045ZINFOtest_unauthorized (InternalApi): Created recovery user's password
16092025-04-30T06:43:43.046ZTRACtest_unauthorized (InternalApi): authorize begin action = ModifyPolicy actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Silo { parent: Fleet, key: f8682c7a-46dd-47d9-8550-62e4b1c3e981, lookup_type: ById(f8682c7a-46dd-47d9-8550-62e4b1c3e981) }
16102025-04-30T06:43:43.046ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = f8682c7a-46dd-47d9-8550-62e4b1c3e981 resource_type = Silo
16112025-04-30T06:43:43.046ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16122025-04-30T06:43:43.046ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
16132025-04-30T06:43:43.046ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16142025-04-30T06:43:43.051ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16152025-04-30T06:43:43.054ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
16162025-04-30T06:43:43.054ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
16172025-04-30T06:43:43.054ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16182025-04-30T06:43:43.054ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
16192025-04-30T06:43:43.054ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
16202025-04-30T06:43:43.088ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
16212025-04-30T06:43:43.121ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16222025-04-30T06:43:43.121ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16232025-04-30T06:43:43.121ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
16242025-04-30T06:43:43.122ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16252025-04-30T06:43:43.123ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16262025-04-30T06:43:43.127ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
16272025-04-30T06:43:43.127ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
16282025-04-30T06:43:43.127ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
16292025-04-30T06:43:43.128ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
16302025-04-30T06:43:43.133ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16312025-04-30T06:43:43.136ZDEBGtest_unauthorized (InternalApi): authorize result action = ModifyPolicy actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Silo { parent: Fleet, key: f8682c7a-46dd-47d9-8550-62e4b1c3e981, lookup_type: ById(f8682c7a-46dd-47d9-8550-62e4b1c3e981) } result = Ok(())
16322025-04-30T06:43:43.136ZDEBGtest_unauthorized (InternalApi): Generated role assignment queries
16332025-04-30T06:43:43.181ZINFOtest_unauthorized (InternalApi): Granted Silo privileges
16342025-04-30T06:43:43.181ZTRACtest_unauthorized (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
16352025-04-30T06:43:43.181ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16362025-04-30T06:43:43.181ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16372025-04-30T06:43:43.181ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
16382025-04-30T06:43:43.182ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16392025-04-30T06:43:43.222ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
16402025-04-30T06:43:43.222ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory
16412025-04-30T06:43:43.223ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:54489 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
16422025-04-30T06:43:43.223ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:54489 req_id = 79d75b21-cf41-446f-b43b-56c9ca4759d8 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
16432025-04-30T06:43:43.224ZINFOtest_unauthorized (dropshot): request completed latency_us = 550 local_addr = [::1]:57786 method = GET remote_addr = [::1]:54489 req_id = 79d75b21-cf41-446f-b43b-56c9ca4759d8 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
16442025-04-30T06:43:43.224ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16452025-04-30T06:43:43.226ZDEBGtest_unauthorized (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
16462025-04-30T06:43:43.227ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = http://[::1]:57786 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:57786/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "79d75b21-cf41-446f-b43b-56c9ca4759d8", "content-length": "2769", "date": "Wed, 30 Apr 2025 06:43:42 GMT"} })
16472025-04-30T06:43:43.228ZDEBGtest_unauthorized (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
16482025-04-30T06:43:43.228ZDEBGtest_unauthorized (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
16492025-04-30T06:43:43.228ZDEBGtest_unauthorized (BackgroundTasks): finished collection background_task = inventory_collection
16502025-04-30T06:43:43.230ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
16512025-04-30T06:43:43.230ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16522025-04-30T06:43:43.230ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
16532025-04-30T06:43:43.230ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
16542025-04-30T06:43:43.230ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
16552025-04-30T06:43:43.255ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16562025-04-30T06:43:43.255ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
16572025-04-30T06:43:43.255ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16582025-04-30T06:43:43.256ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16592025-04-30T06:43:43.259ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
16602025-04-30T06:43:43.259ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
16612025-04-30T06:43:43.259ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
16622025-04-30T06:43:43.260ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
16632025-04-30T06:43:43.270ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16642025-04-30T06:43:43.270ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
16652025-04-30T06:43:43.271ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16662025-04-30T06:43:43.288ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16672025-04-30T06:43:43.288ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
16682025-04-30T06:43:43.288ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16692025-04-30T06:43:43.350ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
16702025-04-30T06:43:43.350ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
16712025-04-30T06:43:43.350ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16722025-04-30T06:43:43.350ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
16732025-04-30T06:43:43.351ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16742025-04-30T06:43:43.381ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
16752025-04-30T06:43:43.381ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16762025-04-30T06:43:43.381ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
16772025-04-30T06:43:43.382ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16782025-04-30T06:43:43.384ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16792025-04-30T06:43:43.384ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16802025-04-30T06:43:43.384ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
16812025-04-30T06:43:43.385ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16822025-04-30T06:43:43.396ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16832025-04-30T06:43:43.409ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
16842025-04-30T06:43:43.409ZTRACtest_unauthorized (InternalApi): successful lookup of siloed resource "Vpc" using built-in user
16852025-04-30T06:43:43.409ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
16862025-04-30T06:43:43.409ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
16872025-04-30T06:43:43.409ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16882025-04-30T06:43:43.409ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
16892025-04-30T06:43:43.409ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16902025-04-30T06:43:43.412ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
16912025-04-30T06:43:43.412ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16922025-04-30T06:43:43.412ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
16932025-04-30T06:43:43.412ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16942025-04-30T06:43:43.427ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16952025-04-30T06:43:43.427ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16962025-04-30T06:43:43.427ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
16972025-04-30T06:43:43.428ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16982025-04-30T06:43:43.433ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16992025-04-30T06:43:43.442ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
17002025-04-30T06:43:43.442ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17012025-04-30T06:43:43.442ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
17022025-04-30T06:43:43.443ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17032025-04-30T06:43:43.448ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17042025-04-30T06:43:43.448ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
17052025-04-30T06:43:43.448ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17062025-04-30T06:43:43.451ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17072025-04-30T06:43:43.451ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
17082025-04-30T06:43:43.452ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17092025-04-30T06:43:43.453ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17102025-04-30T06:43:43.454ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
17112025-04-30T06:43:43.456ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17122025-04-30T06:43:43.470ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
17132025-04-30T06:43:43.470ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
17142025-04-30T06:43:43.470ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17152025-04-30T06:43:43.470ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
17162025-04-30T06:43:43.470ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17172025-04-30T06:43:43.472ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
17182025-04-30T06:43:43.474ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17192025-04-30T06:43:43.474ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
17202025-04-30T06:43:43.475ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17212025-04-30T06:43:43.481ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
17222025-04-30T06:43:43.481ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17232025-04-30T06:43:43.481ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
17242025-04-30T06:43:43.482ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17252025-04-30T06:43:43.503ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17262025-04-30T06:43:43.534ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
17272025-04-30T06:43:43.534ZTRACtest_unauthorized (InternalApi): successful lookup of siloed resource "Vpc" using built-in user
17282025-04-30T06:43:43.534ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17292025-04-30T06:43:43.534ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
17302025-04-30T06:43:43.534ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17312025-04-30T06:43:43.538ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17322025-04-30T06:43:43.538ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
17332025-04-30T06:43:43.540ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17342025-04-30T06:43:43.547ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17352025-04-30T06:43:43.547ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
17362025-04-30T06:43:43.547ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17372025-04-30T06:43:43.561ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17382025-04-30T06:43:43.561ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
17392025-04-30T06:43:43.562ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17402025-04-30T06:43:43.574ZTRACtest_unauthorized (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") }
17412025-04-30T06:43:43.574ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
17422025-04-30T06:43:43.574ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17432025-04-30T06:43:43.574ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
17442025-04-30T06:43:43.574ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17452025-04-30T06:43:43.580ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
17462025-04-30T06:43:43.580ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17472025-04-30T06:43:43.580ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
17482025-04-30T06:43:43.581ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17492025-04-30T06:43:43.586ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
17502025-04-30T06:43:43.586ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17512025-04-30T06:43:43.586ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
17522025-04-30T06:43:43.587ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17532025-04-30T06:43:43.589ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17542025-04-30T06:43:43.599ZDEBGtest_unauthorized (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
17552025-04-30T06:43:43.599ZTRACtest_unauthorized (InternalApi): successful lookup of siloed resource "VpcSubnet" using built-in user
17562025-04-30T06:43:43.599ZTRACtest_unauthorized (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") }
17572025-04-30T06:43:43.599ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
17582025-04-30T06:43:43.599ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17592025-04-30T06:43:43.599ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
17602025-04-30T06:43:43.600ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17612025-04-30T06:43:43.617ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
17622025-04-30T06:43:43.617ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17632025-04-30T06:43:43.617ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
17642025-04-30T06:43:43.618ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17652025-04-30T06:43:43.629ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
17662025-04-30T06:43:43.629ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17672025-04-30T06:43:43.629ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
17682025-04-30T06:43:43.630ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17692025-04-30T06:43:43.641ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17702025-04-30T06:43:43.660ZDEBGtest_unauthorized (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
17712025-04-30T06:43:43.660ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17722025-04-30T06:43:43.660ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
17732025-04-30T06:43:43.661ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17742025-04-30T06:43:43.668ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17752025-04-30T06:43:43.668ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
17762025-04-30T06:43:43.669ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17772025-04-30T06:43:43.676ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17782025-04-30T06:43:43.677ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
17792025-04-30T06:43:43.677ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17802025-04-30T06:43:43.681ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17812025-04-30T06:43:43.681ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
17822025-04-30T06:43:43.682ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17832025-04-30T06:43:43.692ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17842025-04-30T06:43:43.692ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
17852025-04-30T06:43:43.693ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17862025-04-30T06:43:43.698ZTRACtest_unauthorized (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") }
17872025-04-30T06:43:43.698ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
17882025-04-30T06:43:43.698ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17892025-04-30T06:43:43.698ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
17902025-04-30T06:43:43.699ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17912025-04-30T06:43:43.703ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
17922025-04-30T06:43:43.703ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17932025-04-30T06:43:43.703ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
17942025-04-30T06:43:43.704ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17952025-04-30T06:43:43.717ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
17962025-04-30T06:43:43.718ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17972025-04-30T06:43:43.718ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
17982025-04-30T06:43:43.718ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17992025-04-30T06:43:43.729ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18002025-04-30T06:43:43.738ZDEBGtest_unauthorized (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
18012025-04-30T06:43:43.738ZTRACtest_unauthorized (InternalApi): successful lookup of siloed resource "VpcSubnet" using built-in user
18022025-04-30T06:43:43.738ZTRACtest_unauthorized (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") }
18032025-04-30T06:43:43.738ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
18042025-04-30T06:43:43.738ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18052025-04-30T06:43:43.739ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
18062025-04-30T06:43:43.740ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18072025-04-30T06:43:43.743ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
18082025-04-30T06:43:43.744ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18092025-04-30T06:43:43.744ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
18102025-04-30T06:43:43.744ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18112025-04-30T06:43:43.748ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
18122025-04-30T06:43:43.748ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18132025-04-30T06:43:43.748ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
18142025-04-30T06:43:43.748ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18152025-04-30T06:43:43.754ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18162025-04-30T06:43:43.762ZDEBGtest_unauthorized (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
18172025-04-30T06:43:43.762ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18182025-04-30T06:43:43.762ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
18192025-04-30T06:43:43.763ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18202025-04-30T06:43:43.806ZDEBGtest_unauthorized (nexus): resolved names for firewall rules instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 3466d994-2ae5-4a93-8b25-2ea17242d33b, kind: Service { id: 9a892c5e-6788-4649-8d0f-08ee55eaa359 }, name: Name("external-dns-9a892c5e-6788-4649-8d0f-08ee55eaa359"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 0])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: 5036fcc9-7261-43e3-9e27-69fb4ef2e46e, kind: Service { id: 8d752adb-08b9-405f-a11b-66019bf447b5 }, name: Name("nexus-8d752adb-08b9-405f-a11b-66019bf447b5"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 1])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
18212025-04-30T06:43:43.806ZTRACtest_unauthorized (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
18222025-04-30T06:43:43.806ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
18232025-04-30T06:43:43.806ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18242025-04-30T06:43:43.806ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
18252025-04-30T06:43:43.808ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18262025-04-30T06:43:43.811ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18272025-04-30T06:43:43.818ZDEBGtest_unauthorized (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
18282025-04-30T06:43:43.818ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18292025-04-30T06:43:43.818ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
18302025-04-30T06:43:43.818ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18312025-04-30T06:43:43.823ZTRACtest_unauthorized (nexus): fetched allowlist from DB allowed = AllowList { id: 001de000-a110-4000-8000-000000000000, time_created: 2025-04-30T06:06:08.248500Z, time_modified: 2025-04-30T06:43:43.226964Z, allowed_ips: None }
18322025-04-30T06:43:43.823ZDEBGtest_unauthorized (nexus): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified.
18332025-04-30T06:43:43.824ZDEBGtest_unauthorized (nexus): resolved firewall rules for sled agents sled_agent_rules = [ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 3466d994-2ae5-4a93-8b25-2ea17242d33b, kind: Service { id: 9a892c5e-6788-4649-8d0f-08ee55eaa359 }, name: Name("external-dns-9a892c5e-6788-4649-8d0f-08ee55eaa359"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 0])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }, ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 5036fcc9-7261-43e3-9e27-69fb4ef2e46e, kind: Service { id: 8d752adb-08b9-405f-a11b-66019bf447b5 }, name: Name("nexus-8d752adb-08b9-405f-a11b-66019bf447b5"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 1])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }]
18342025-04-30T06:43:43.824ZDEBGtest_unauthorized (nexus): resolved 2 rules for sleds
18352025-04-30T06:43:43.834ZINFOtest_unauthorized (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 1a8b2318-abf8-406d-a389-112ab91a0de8
18362025-04-30T06:43:43.834ZDEBGtest_unauthorized (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 1a8b2318-abf8-406d-a389-112ab91a0de8 time_started = 2025-04-30 06:43:41.812367 UTC
18372025-04-30T06:43:43.834ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 6.384897492s iteration = 2 status = {"collection_id":"1a8b2318-abf8-406d-a389-112ab91a0de8","time_done":"2025-04-30 06:43:43.228755 UTC","time_started":"2025-04-30 06:43:41.812367 UTC"}
18382025-04-30T06:43:43.834ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 2 reason = Dependency
18392025-04-30T06:43:43.834ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 44.869\u{b5}s iteration = 2 status = {"error":"task disabled"}
18402025-04-30T06:43:43.834ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 2 reason = Dependency
18412025-04-30T06:43:43.834ZWARNtest_unauthorized (BackgroundTasks): Blueprint rendezvous: skipped background_task = blueprint_rendezvous reason = no blueprint
18422025-04-30T06:43:43.834ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 18.376\u{b5}s iteration = 2 status = {"error":"no blueprint"}
18432025-04-30T06:43:43.858ZDEBGtest_unauthorized (nexus): resolved sleds for vpc oxide-services vpc_to_sled = [Sled { identity: SledIdentity { id: b6d65341-167c-41df-9b5c-41cded99c229, time_created: 2025-04-30T06:43:36.474963Z, time_modified: 2025-04-30T06:43:36.474963Z }, time_deleted: None, rcgen: Generation(Generation(1)), rack_id: c19a698f-c6f9-4a17-ae30-20d711b8f7dc, is_scrimlet: true, serial_number: "sim-b6d65341", part_number: "Unknown", revision: SqlU32(0), usable_hardware_threads: SqlU32(16), usable_physical_ram: ByteCount(ByteCount(34359738368)), reservoir_size: ByteCount(ByteCount(17179869184)), ip: ::1, port: SqlU16(57786), last_used_address: ::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(48692) }]
18442025-04-30T06:43:43.901ZDEBGtest_unauthorized (nexus): sending firewall rules to sled agents
18452025-04-30T06:43:43.902ZTRACtest_unauthorized (InstanceAllocator): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database
18462025-04-30T06:43:43.902ZDEBGtest_unauthorized (InstanceAllocator): roles roles = RoleSet { roles: {} }
18472025-04-30T06:43:43.903ZDEBGtest_unauthorized (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
18482025-04-30T06:43:43.920ZTRACtest_unauthorized (InstanceAllocator): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Sled { parent: Fleet, key: b6d65341-167c-41df-9b5c-41cded99c229, lookup_type: ById(b6d65341-167c-41df-9b5c-41cded99c229) }
18492025-04-30T06:43:43.920ZTRACtest_unauthorized (InstanceAllocator): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
18502025-04-30T06:43:43.920ZTRACtest_unauthorized (InstanceAllocator): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database
18512025-04-30T06:43:43.920ZDEBGtest_unauthorized (InstanceAllocator): roles roles = RoleSet { roles: {} }
18522025-04-30T06:43:43.921ZDEBGtest_unauthorized (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
18532025-04-30T06:43:43.936ZDEBGtest_unauthorized (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
18542025-04-30T06:43:43.948ZDEBGtest_unauthorized (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Sled { parent: Fleet, key: b6d65341-167c-41df-9b5c-41cded99c229, lookup_type: ById(b6d65341-167c-41df-9b5c-41cded99c229) } result = Ok(())
18552025-04-30T06:43:43.991ZDEBGtest_unauthorized (nexus): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 body = Some(Body) method = PUT uri = http://[::1]:57786/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
18562025-04-30T06:43:43.993ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:63421 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
18572025-04-30T06:43:43.994ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = PUT remote_addr = [::1]:63421 req_id = f532585f-ddcc-4de3-aea5-da2c7679038a sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
18582025-04-30T06:43:43.996ZINFOtest_unauthorized (dropshot): request completed latency_us = 1770 local_addr = [::1]:57786 method = PUT remote_addr = [::1]:63421 req_id = f532585f-ddcc-4de3-aea5-da2c7679038a response_code = 204 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
18592025-04-30T06:43:43.996ZDEBGtest_unauthorized (nexus): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 result = Ok(Response { url: "http://[::1]:57786/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "f532585f-ddcc-4de3-aea5-da2c7679038a", "date": "Wed, 30 Apr 2025 06:43:43 GMT"} })
18602025-04-30T06:43:43.997ZINFOtest_unauthorized (nexus): updated firewall rules on 1 sleds
18612025-04-30T06:43:43.997ZTRACtest_unauthorized (ServiceBalancer): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database
18622025-04-30T06:43:43.997ZDEBGtest_unauthorized (ServiceBalancer): roles roles = RoleSet { roles: {} }
18632025-04-30T06:43:43.998ZDEBGtest_unauthorized (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
18642025-04-30T06:43:43.998ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 2 reason = Signaled
18652025-04-30T06:43:43.998ZDEBGtest_unauthorized (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
18662025-04-30T06:43:43.998ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
18672025-04-30T06:43:43.998ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
18682025-04-30T06:43:43.999ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
18692025-04-30T06:43:43.999ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 2 reason = Signaled
18702025-04-30T06:43:43.999ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal
18712025-04-30T06:43:43.999ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 2 reason = Signaled
18722025-04-30T06:43:43.999ZDEBGtest_unauthorized (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
18732025-04-30T06:43:43.999ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
18742025-04-30T06:43:43.999ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
18752025-04-30T06:43:43.999ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
18762025-04-30T06:43:44.000ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 2 reason = Signaled
18772025-04-30T06:43:44.000ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal
18782025-04-30T06:43:44.000ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = external_endpoints iteration = 2 reason = Signaled
18792025-04-30T06:43:44.000ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet
18802025-04-30T06:43:44.000ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = external_endpoints resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
18812025-04-30T06:43:44.000ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
18822025-04-30T06:43:44.000ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
18832025-04-30T06:43:44.001ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
18842025-04-30T06:43:44.001ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 3 reason = Signaled
18852025-04-30T06:43:44.001ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
18862025-04-30T06:43:44.001ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
18872025-04-30T06:43:44.001ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
18882025-04-30T06:43:44.001ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
18892025-04-30T06:43:44.001ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
18902025-04-30T06:43:44.002ZTRACtest_unauthorized (dns): buffer buffer = Length: 75 (0x4b) bytes\n0000: d5 2e 01 00 00 01 00 00 00 00 00 01 0c 5f 6e 61 ............._na\n0010: 6d 65 73 65 72 76 69 63 65 04 5f 74 63 70 0d 63 meservice._tcp.c\n0020: 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 ontrol-plane.oxi\n0030: 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 de.internal..!..\n0040: 00 00 29 04 d0 00 00 00 00 00 00 ..)........ kind = dns peer_addr = [::1]:56356 req_id = 36b7d2a1-f2ba-48fa-b6e8-2c45fe08e1cd
18912025-04-30T06:43:44.002ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 54574,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_nameservice._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_nameservice._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 12,\n 95,\n 110,\n 97,\n 109,\n 101,\n 115,\n 101,\n 114,\n 118,\n 105,\n 99,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:56356 req_id = 36b7d2a1-f2ba-48fa-b6e8-2c45fe08e1cd
18922025-04-30T06:43:44.002ZDEBGtest_unauthorized (store): query key key = _nameservice._tcp
18932025-04-30T06:43:44.003ZDEBGtest_unauthorized (store): query key key = f74fcbb5-b3b5-4928-b67d-87dd9b2f844a.host
18942025-04-30T06:43:44.003ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("f74fcbb5-b3b5-4928-b67d-87dd9b2f844a.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:56356 query = LowerQuery { name: LowerName(Name("_nameservice._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 39462, target: Name("f74fcbb5-b3b5-4928-b67d-87dd9b2f844a.host.control-plane.oxide.internal") })) }] req_id = 36b7d2a1-f2ba-48fa-b6e8-2c45fe08e1cd
18952025-04-30T06:43:44.004ZTRACtest_unauthorized (dns): buffer buffer = Length: 76 (0x4c) bytes\n0000: b8 2a 01 00 00 01 00 00 00 00 00 01 0d 5f 65 78 .*..........._ex\n0010: 74 65 72 6e 61 6c 2d 64 6e 73 04 5f 74 63 70 0d ternal-dns._tcp.\n0020: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0030: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 21 00 ide.internal..!.\n0040: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:55039 req_id = cba6135c-8595-4509-8ff7-03b6963eea35
18962025-04-30T06:43:44.004ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 47146,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_external-dns._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_external-dns._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 13,\n 95,\n 101,\n 120,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 45,\n 100,\n 110,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:55039 req_id = cba6135c-8595-4509-8ff7-03b6963eea35
18972025-04-30T06:43:44.004ZDEBGtest_unauthorized (store): query key key = _external-dns._tcp
18982025-04-30T06:43:44.004ZDEBGtest_unauthorized (store): query key key = 9a892c5e-6788-4649-8d0f-08ee55eaa359.host
18992025-04-30T06:43:44.004ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("9a892c5e-6788-4649-8d0f-08ee55eaa359.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:55039 query = LowerQuery { name: LowerName(Name("_external-dns._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 61288, target: Name("9a892c5e-6788-4649-8d0f-08ee55eaa359.host.control-plane.oxide.internal") })) }] req_id = cba6135c-8595-4509-8ff7-03b6963eea35
19002025-04-30T06:43:44.005ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 39462, target: Name("f74fcbb5-b3b5-4928-b67d-87dd9b2f844a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("f74fcbb5-b3b5-4928-b67d-87dd9b2f844a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4288, tv_nsec: 804167129 } })
19012025-04-30T06:43:44.005ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
19022025-04-30T06:43:44.005ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 61288, target: Name("9a892c5e-6788-4649-8d0f-08ee55eaa359.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("9a892c5e-6788-4649-8d0f-08ee55eaa359.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4288, tv_nsec: 804491475 } })
19032025-04-30T06:43:44.005ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
19042025-04-30T06:43:44.005ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig
19052025-04-30T06:43:44.005ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_internal dns_group = internal resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19062025-04-30T06:43:44.005ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
19072025-04-30T06:43:44.006ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
19082025-04-30T06:43:44.006ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
19092025-04-30T06:43:44.006ZTRACtest_unauthorized (ServiceBalancer): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: c19a698f-c6f9-4a17-ae30-20d711b8f7dc, lookup_type: ById(c19a698f-c6f9-4a17-ae30-20d711b8f7dc) }
19102025-04-30T06:43:44.006ZTRACtest_unauthorized (ServiceBalancer): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19112025-04-30T06:43:44.006ZTRACtest_unauthorized (ServiceBalancer): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database
19122025-04-30T06:43:44.006ZDEBGtest_unauthorized (ServiceBalancer): roles roles = RoleSet { roles: {} }
19132025-04-30T06:43:44.007ZDEBGtest_unauthorized (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
19142025-04-30T06:43:44.007ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig
19152025-04-30T06:43:44.007ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_external dns_group = external resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19162025-04-30T06:43:44.007ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
19172025-04-30T06:43:44.007ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
19182025-04-30T06:43:44.008ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
19192025-04-30T06:43:44.008ZTRACtest_unauthorized (dns): buffer buffer = Length: 99 (0x63) bytes\n0000: aa 57 01 00 00 01 00 00 00 00 00 01 24 66 37 34 .W..........$f74\n0010: 66 63 62 62 35 2d 62 33 62 35 2d 34 39 32 38 2d fcbb5-b3b5-4928-\n0020: 62 36 37 64 2d 38 37 64 64 39 62 32 66 38 34 34 b67d-87dd9b2f844\n0030: 61 04 68 6f 73 74 0d 63 6f 6e 74 72 6f 6c 2d 70 a.host.control-p\n0040: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0050: 6e 61 6c 00 00 1c 00 01 00 00 29 04 d0 00 00 00 nal.......).....\n0060: 00 00 00 ... kind = dns peer_addr = [::1]:50451 req_id = e40a358a-0080-414d-9e1c-f162195be2cd
19202025-04-30T06:43:44.008ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 43607,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("f74fcbb5-b3b5-4928-b67d-87dd9b2f844a.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("f74fcbb5-b3b5-4928-b67d-87dd9b2f844a.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 36,\n 102,\n 55,\n 52,\n 102,\n 99,\n 98,\n 98,\n 53,\n 45,\n 98,\n 51,\n 98,\n 53,\n 45,\n 52,\n 57,\n 50,\n 56,\n 45,\n 98,\n 54,\n 55,\n 100,\n 45,\n 56,\n 55,\n 100,\n 100,\n 57,\n 98,\n 50,\n 102,\n 56,\n 52,\n 52,\n 97,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:50451 req_id = e40a358a-0080-414d-9e1c-f162195be2cd
19212025-04-30T06:43:44.009ZDEBGtest_unauthorized (store): query key key = f74fcbb5-b3b5-4928-b67d-87dd9b2f844a.host
19222025-04-30T06:43:44.009ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:50451 query = LowerQuery { name: LowerName(Name("f74fcbb5-b3b5-4928-b67d-87dd9b2f844a.host.control-plane.oxide.internal.")), original: Query { name: Name("f74fcbb5-b3b5-4928-b67d-87dd9b2f844a.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("f74fcbb5-b3b5-4928-b67d-87dd9b2f844a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = e40a358a-0080-414d-9e1c-f162195be2cd
19232025-04-30T06:43:44.009ZTRACtest_unauthorized (dns): buffer buffer = Length: 99 (0x63) bytes\n0000: 5d 6b 01 00 00 01 00 00 00 00 00 01 24 39 61 38 ]k..........$9a8\n0010: 39 32 63 35 65 2d 36 37 38 38 2d 34 36 34 39 2d 92c5e-6788-4649-\n0020: 38 64 30 66 2d 30 38 65 65 35 35 65 61 61 33 35 8d0f-08ee55eaa35\n0030: 39 04 68 6f 73 74 0d 63 6f 6e 74 72 6f 6c 2d 70 9.host.control-p\n0040: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0050: 6e 61 6c 00 00 1c 00 01 00 00 29 04 d0 00 00 00 nal.......).....\n0060: 00 00 00 ... kind = dns peer_addr = [::1]:60356 req_id = 5da54cca-9def-4cf9-acd7-e8efaa8c69eb
19242025-04-30T06:43:44.009ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 23915,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("9a892c5e-6788-4649-8d0f-08ee55eaa359.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("9a892c5e-6788-4649-8d0f-08ee55eaa359.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 36,\n 57,\n 97,\n 56,\n 57,\n 50,\n 99,\n 53,\n 101,\n 45,\n 54,\n 55,\n 56,\n 56,\n 45,\n 52,\n 54,\n 52,\n 57,\n 45,\n 56,\n 100,\n 48,\n 102,\n 45,\n 48,\n 56,\n 101,\n 101,\n 53,\n 53,\n 101,\n 97,\n 97,\n 51,\n 53,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60356 req_id = 5da54cca-9def-4cf9-acd7-e8efaa8c69eb
19252025-04-30T06:43:44.009ZDEBGtest_unauthorized (store): query key key = 9a892c5e-6788-4649-8d0f-08ee55eaa359.host
19262025-04-30T06:43:44.009ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:60356 query = LowerQuery { name: LowerName(Name("9a892c5e-6788-4649-8d0f-08ee55eaa359.host.control-plane.oxide.internal.")), original: Query { name: Name("9a892c5e-6788-4649-8d0f-08ee55eaa359.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("9a892c5e-6788-4649-8d0f-08ee55eaa359.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 5da54cca-9def-4cf9-acd7-e8efaa8c69eb
19272025-04-30T06:43:44.010ZINFOtest_unauthorized (BackgroundTasks): found DNS servers (initial) addresses = DnsServersList { addresses: [[::1]:39462] } background_task = dns_servers_internal dns_group = internal
19282025-04-30T06:43:44.010ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 10.865879ms iteration = 2 status = {"addresses":["[::1]:39462"]}
19292025-04-30T06:43:44.010ZINFOtest_unauthorized (BackgroundTasks): found DNS servers (initial) addresses = DnsServersList { addresses: [[::1]:61288] } background_task = dns_servers_external dns_group = external
19302025-04-30T06:43:44.010ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 10.371785ms iteration = 2 status = {"addresses":["[::1]:61288"]}
19312025-04-30T06:43:44.010ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 2 reason = Dependency
19322025-04-30T06:43:44.010ZWARNtest_unauthorized (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_internal dns_group = internal reason = no config
19332025-04-30T06:43:44.010ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 22.985\u{b5}s iteration = 2 status = {"error":"no config"}
19342025-04-30T06:43:44.010ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 2 reason = Dependency
19352025-04-30T06:43:44.010ZWARNtest_unauthorized (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_external dns_group = external reason = no config
19362025-04-30T06:43:44.010ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 9.759\u{b5}s iteration = 2 status = {"error":"no config"}
19372025-04-30T06:43:44.011ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19382025-04-30T06:43:44.016ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
19392025-04-30T06:43:44.016ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
19402025-04-30T06:43:44.016ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
19412025-04-30T06:43:44.016ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
19422025-04-30T06:43:44.017ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19432025-04-30T06:43:44.025ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
19442025-04-30T06:43:44.025ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
19452025-04-30T06:43:44.025ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
19462025-04-30T06:43:44.026ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
19472025-04-30T06:43:44.026ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19482025-04-30T06:43:44.035ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
19492025-04-30T06:43:44.038ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19502025-04-30T06:43:44.044ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
19512025-04-30T06:43:44.045ZDEBGtest_unauthorized (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
19522025-04-30T06:43:44.045ZDEBGtest_unauthorized (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
19532025-04-30T06:43:44.045ZDEBGtest_unauthorized (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
19542025-04-30T06:43:44.045ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig
19552025-04-30T06:43:44.045ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_internal dns_group = internal resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19562025-04-30T06:43:44.045ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
19572025-04-30T06:43:44.045ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
19582025-04-30T06:43:44.046ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
19592025-04-30T06:43:44.046ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig
19602025-04-30T06:43:44.046ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = external_endpoints resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19612025-04-30T06:43:44.046ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
19622025-04-30T06:43:44.046ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
19632025-04-30T06:43:44.048ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
19642025-04-30T06:43:44.056ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1a8b2318-abf8-406d-a389-112ab91a0de8, 0)]
19652025-04-30T06:43:44.056ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
19662025-04-30T06:43:44.058ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19672025-04-30T06:43:44.067ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
19682025-04-30T06:43:44.067ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
19692025-04-30T06:43:44.067ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
19702025-04-30T06:43:44.068ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
19712025-04-30T06:43:44.069ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19722025-04-30T06:43:44.075ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
19732025-04-30T06:43:44.076ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
19742025-04-30T06:43:44.076ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
19752025-04-30T06:43:44.076ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
19762025-04-30T06:43:44.077ZDEBGtest_unauthorized (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19772025-04-30T06:43:44.087ZDEBGtest_unauthorized (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: c19a698f-c6f9-4a17-ae30-20d711b8f7dc, lookup_type: ById(c19a698f-c6f9-4a17-ae30-20d711b8f7dc) } result = Ok(())
19782025-04-30T06:43:44.089ZINFOtest_unauthorized (nexus): Rack initialized
19792025-04-30T06:43:44.089ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
19802025-04-30T06:43:44.089ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
19812025-04-30T06:43:44.090ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
19822025-04-30T06:43:44.091ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 74 76 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 tv..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:60413 req_id = aca795d0-9c3d-446b-b21c-6b6f132fffbe
19832025-04-30T06:43:44.091ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 29814,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60413 req_id = aca795d0-9c3d-446b-b21c-6b6f132fffbe
19842025-04-30T06:43:44.092ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
19852025-04-30T06:43:44.092ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
19862025-04-30T06:43:44.092ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:60413 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = aca795d0-9c3d-446b-b21c-6b6f132fffbe
19872025-04-30T06:43:44.094ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4288, tv_nsec: 892906296 } })
19882025-04-30T06:43:44.094ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
19892025-04-30T06:43:44.097ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 7b fb 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e {...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:53357 req_id = 7b1fafee-4a96-47b7-aecf-127e431694f2
19902025-04-30T06:43:44.097ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 31739,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:53357 req_id = 7b1fafee-4a96-47b7-aecf-127e431694f2
19912025-04-30T06:43:44.097ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
19922025-04-30T06:43:44.097ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:53357 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 7b1fafee-4a96-47b7-aecf-127e431694f2
19932025-04-30T06:43:44.155ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
19942025-04-30T06:43:44.156ZDEBGtest_unauthorized (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
19952025-04-30T06:43:44.156ZDEBGtest_unauthorized (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 9b32c876-c6b8-4517-9af1-ab0183012dc4 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
19962025-04-30T06:43:44.156ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig
19972025-04-30T06:43:44.156ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_internal dns_group = internal resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19982025-04-30T06:43:44.156ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
19992025-04-30T06:43:44.156ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
20002025-04-30T06:43:44.157ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
20012025-04-30T06:43:44.158ZDEBGtest_unauthorized (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
20022025-04-30T06:43:44.158ZDEBGtest_unauthorized (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
20032025-04-30T06:43:44.158ZDEBGtest_unauthorized (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
20042025-04-30T06:43:44.158ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig
20052025-04-30T06:43:44.158ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_external dns_group = external resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
20062025-04-30T06:43:44.158ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
20072025-04-30T06:43:44.158ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
20082025-04-30T06:43:44.158ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
20092025-04-30T06:43:44.159ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet
20102025-04-30T06:43:44.159ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = external_endpoints resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
20112025-04-30T06:43:44.159ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
20122025-04-30T06:43:44.159ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
20132025-04-30T06:43:44.159ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
20142025-04-30T06:43:44.162ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20152025-04-30T06:43:44.169ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
20162025-04-30T06:43:44.169ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
20172025-04-30T06:43:44.169ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
20182025-04-30T06:43:44.169ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
20192025-04-30T06:43:44.171ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20202025-04-30T06:43:44.176ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
20212025-04-30T06:43:44.176ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
20222025-04-30T06:43:44.176ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
20232025-04-30T06:43:44.177ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
20242025-04-30T06:43:44.177ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20252025-04-30T06:43:44.184ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
20262025-04-30T06:43:44.184ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
20272025-04-30T06:43:44.184ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
20282025-04-30T06:43:44.185ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
20292025-04-30T06:43:44.186ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
20302025-04-30T06:43:44.186ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
20312025-04-30T06:43:44.187ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
20322025-04-30T06:43:44.200ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: ec 8b 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:55677 req_id = 43da1c3d-f03b-44b0-a813-9a09abdee082
20332025-04-30T06:43:44.200ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 60555,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:55677 req_id = 43da1c3d-f03b-44b0-a813-9a09abdee082
20342025-04-30T06:43:44.201ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
20352025-04-30T06:43:44.201ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 60555, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:55677 req_id = 43da1c3d-f03b-44b0-a813-9a09abdee082
20362025-04-30T06:43:44.203ZDEBGtest_unauthorized (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 9b32c876-c6b8-4517-9af1-ab0183012dc4 dns_zone_name = control-plane.oxide.internal found_so_far = 21 version = 1
20372025-04-30T06:43:44.203ZDEBGtest_unauthorized (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
20382025-04-30T06:43:44.203ZINFOtest_unauthorized (BackgroundTasks): found latest generation (first find) background_task = dns_config_internal dns_group = internal generation = 1
20392025-04-30T06:43:44.203ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 204.936773ms iteration = 2 status = {"generation":1}
20402025-04-30T06:43:44.203ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 3 reason = Dependency
20412025-04-30T06:43:44.270ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[::1]:39462 generation = 1 method = PUT servers = DnsServersList { addresses: [[::1]:39462] } uri = http://[::1]:39462/config
20422025-04-30T06:43:44.271ZDEBGtest_unauthorized (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
20432025-04-30T06:43:44.271ZDEBGtest_unauthorized (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 2b03552f-f865-41a5-9e06-eedae12fe0fd dns_zone_name = oxide-dev.test found_so_far = 0 version = 2
20442025-04-30T06:43:44.271ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig
20452025-04-30T06:43:44.271ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_external dns_group = external resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
20462025-04-30T06:43:44.271ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
20472025-04-30T06:43:44.272ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
20482025-04-30T06:43:44.273ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
20492025-04-30T06:43:44.275ZINFOtest_unauthorized (BackgroundTasks): found Silo/DNS/TLS config (initial) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide-dev.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-04-30T06:43:34.574956Z, time_modified: 2025-04-30T06:43:34.574956Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "test-suite-silo.sys.oxide-dev.test": ExternalEndpoint { silo_id: f8682c7a-46dd-47d9-8550-62e4b1c3e981, db_silo: Silo { identity: SiloIdentity { id: f8682c7a-46dd-47d9-8550-62e4b1c3e981, name: Name(Name("test-suite-silo")), description: "built-in recovery Silo", time_created: 2025-04-30T06:43:42.951116Z, time_modified: 2025-04-30T06:43:42.951116Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(1)) }, tls_certs: [] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide-dev.test" }, NoSiloCerts { silo_id: f8682c7a-46dd-47d9-8550-62e4b1c3e981, dns_name: "test-suite-silo.sys.oxide-dev.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: f8682c7a-46dd-47d9-8550-62e4b1c3e981, db_silo: Silo { identity: SiloIdentity { id: f8682c7a-46dd-47d9-8550-62e4b1c3e981, name: Name(Name("test-suite-silo")), description: "built-in recovery Silo", time_created: 2025-04-30T06:43:42.951116Z, time_modified: 2025-04-30T06:43:42.951116Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(1)) }, tls_certs: [] }) }
20502025-04-30T06:43:44.275ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 275.054909ms iteration = 2 status = {"by_dns_name":{"default-silo.sys.oxide-dev.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"test-suite-silo.sys.oxide-dev.test":{"silo_id":"f8682c7a-46dd-47d9-8550-62e4b1c3e981","tls_certs":[]}},"default_endpoint":{"silo_id":"f8682c7a-46dd-47d9-8550-62e4b1c3e981","tls_certs":[]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide-dev.test\\" has no usable certificates","silo f8682c7a-46dd-47d9-8550-62e4b1c3e981 with DNS name \\"test-suite-silo.sys.oxide-dev.test\\" has no usable certificates"]}
20512025-04-30T06:43:44.276ZINFOtest_unauthorized (http): accepted connection kind = dns local_addr = [::1]:39462 remote_addr = [::1]:45355
20522025-04-30T06:43:44.276ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
20532025-04-30T06:43:44.276ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
20542025-04-30T06:43:44.276ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
20552025-04-30T06:43:44.277ZTRACtest_unauthorized (dns): buffer buffer = Length: 79 (0x4f) bytes\n0000: d6 18 01 00 00 01 00 00 00 00 00 01 10 5f 63 72 ............._cr\n0010: 75 63 69 62 6c 65 2d 70 61 6e 74 72 79 04 5f 74 ucible-pantry._t\n0020: 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 cp.control-plane\n0030: 05 6f 78 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 .oxide.internal.\n0040: 00 21 00 01 00 00 29 04 d0 00 00 00 00 00 00 .!....)........ kind = dns peer_addr = [::1]:64201 req_id = e388954b-2741-4afe-81ff-07fae4437d3e
20562025-04-30T06:43:44.277ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 54808,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_crucible-pantry._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_crucible-pantry._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 16,\n 95,\n 99,\n 114,\n 117,\n 99,\n 105,\n 98,\n 108,\n 101,\n 45,\n 112,\n 97,\n 110,\n 116,\n 114,\n 121,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64201 req_id = e388954b-2741-4afe-81ff-07fae4437d3e
20572025-04-30T06:43:44.277ZDEBGtest_unauthorized (store): query key key = _crucible-pantry._tcp
20582025-04-30T06:43:44.277ZDEBGtest_unauthorized (store): query key key = fad54123-a622-440a-b83e-b1328fb8bb68.host
20592025-04-30T06:43:44.277ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("fad54123-a622-440a-b83e-b1328fb8bb68.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:64201 query = LowerQuery { name: LowerName(Name("_crucible-pantry._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_crucible-pantry._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_crucible-pantry._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 41451, target: Name("fad54123-a622-440a-b83e-b1328fb8bb68.host.control-plane.oxide.internal") })) }] req_id = e388954b-2741-4afe-81ff-07fae4437d3e
20602025-04-30T06:43:44.278ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20612025-04-30T06:43:44.284ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
20622025-04-30T06:43:44.284ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
20632025-04-30T06:43:44.284ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
20642025-04-30T06:43:44.285ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
20652025-04-30T06:43:44.286ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
20662025-04-30T06:43:44.286ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
20672025-04-30T06:43:44.286ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
20682025-04-30T06:43:44.287ZTRACtest_unauthorized (http): incoming request kind = dns local_addr = [::1]:39462 method = PUT remote_addr = [::1]:45355 req_id = 207537c5-cb5c-4895-becf-b915a6f77f99 uri = /config
20692025-04-30T06:43:44.288ZINFOtest_unauthorized (store): attempting generation update new_generation = 1 req_id = 207537c5-cb5c-4895-becf-b915a6f77f99
20702025-04-30T06:43:44.288ZINFOtest_unauthorized (store): updated generation new_generation = 1 req_id = 207537c5-cb5c-4895-becf-b915a6f77f99
20712025-04-30T06:43:44.289ZINFOtest_unauthorized (http): request completed kind = dns latency_us = 2021 local_addr = [::1]:39462 method = PUT remote_addr = [::1]:45355 req_id = 207537c5-cb5c-4895-becf-b915a6f77f99 response_code = 204 uri = /config
20722025-04-30T06:43:44.290ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[::1]:39462 generation = 1 result = Ok(Response { url: "http://[::1]:39462/config", status: 204, headers: {"x-request-id": "207537c5-cb5c-4895-becf-b915a6f77f99", "date": "Wed, 30 Apr 2025 06:43:43 GMT"} }) servers = DnsServersList { addresses: [[::1]:39462] }
20732025-04-30T06:43:44.290ZINFOtest_unauthorized (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[::1]:39462 generation = 1 servers = DnsServersList { addresses: [[::1]:39462] }
20742025-04-30T06:43:44.292ZINFOtest_unauthorized (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal generation = 1 servers = DnsServersList { addresses: [[::1]:39462] }
20752025-04-30T06:43:44.292ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 89.317339ms iteration = 3 status = {"generation":1,"server_results":{"[::1]:39462":{"Ok":null}}}
20762025-04-30T06:43:44.293ZDEBGtest_unauthorized (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 2b03552f-f865-41a5-9e06-eedae12fe0fd dns_zone_name = oxide-dev.test found_so_far = 1 version = 2
20772025-04-30T06:43:44.293ZDEBGtest_unauthorized (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
20782025-04-30T06:43:44.294ZINFOtest_unauthorized (BackgroundTasks): found latest generation (first find) background_task = dns_config_external dns_group = external generation = 2
20792025-04-30T06:43:44.294ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 294.66971ms iteration = 2 status = {"generation":2}
20802025-04-30T06:43:44.294ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 3 reason = Dependency
20812025-04-30T06:43:44.360ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[::1]:61288 generation = 2 method = PUT servers = DnsServersList { addresses: [[::1]:61288] } uri = http://[::1]:61288/config
20822025-04-30T06:43:44.361ZINFOtest_unauthorized (http): accepted connection kind = dns local_addr = [::1]:61288 remote_addr = [::1]:40820
20832025-04-30T06:43:44.362ZTRACtest_unauthorized (dns): buffer buffer = Length: 99 (0x63) bytes\n0000: a9 3e 01 00 00 01 00 00 00 00 00 01 24 66 61 64 .>..........$fad\n0010: 35 34 31 32 33 2d 61 36 32 32 2d 34 34 30 61 2d 54123-a622-440a-\n0020: 62 38 33 65 2d 62 31 33 32 38 66 62 38 62 62 36 b83e-b1328fb8bb6\n0030: 38 04 68 6f 73 74 0d 63 6f 6e 74 72 6f 6c 2d 70 8.host.control-p\n0040: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0050: 6e 61 6c 00 00 1c 00 01 00 00 29 04 d0 00 00 00 nal.......).....\n0060: 00 00 00 ... kind = dns peer_addr = [::1]:59011 req_id = 688ad3fc-54ed-4550-8171-cee9fa7cfd6e
20842025-04-30T06:43:44.362ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 43326,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("fad54123-a622-440a-b83e-b1328fb8bb68.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("fad54123-a622-440a-b83e-b1328fb8bb68.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 36,\n 102,\n 97,\n 100,\n 53,\n 52,\n 49,\n 50,\n 51,\n 45,\n 97,\n 54,\n 50,\n 50,\n 45,\n 52,\n 52,\n 48,\n 97,\n 45,\n 98,\n 56,\n 51,\n 101,\n 45,\n 98,\n 49,\n 51,\n 50,\n 56,\n 102,\n 98,\n 56,\n 98,\n 98,\n 54,\n 56,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:59011 req_id = 688ad3fc-54ed-4550-8171-cee9fa7cfd6e
20852025-04-30T06:43:44.362ZDEBGtest_unauthorized (store): query key key = fad54123-a622-440a-b83e-b1328fb8bb68.host
20862025-04-30T06:43:44.362ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:59011 query = LowerQuery { name: LowerName(Name("fad54123-a622-440a-b83e-b1328fb8bb68.host.control-plane.oxide.internal.")), original: Query { name: Name("fad54123-a622-440a-b83e-b1328fb8bb68.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("fad54123-a622-440a-b83e-b1328fb8bb68.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 688ad3fc-54ed-4550-8171-cee9fa7cfd6e
20872025-04-30T06:43:44.363ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
20882025-04-30T06:43:44.363ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
20892025-04-30T06:43:44.363ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
20902025-04-30T06:43:44.364ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
20912025-04-30T06:43:44.365ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
20922025-04-30T06:43:44.366ZTRACtest_unauthorized (http): incoming request kind = dns local_addr = [::1]:61288 method = PUT remote_addr = [::1]:40820 req_id = b50ea46e-7bb5-4f5d-8e9e-eedafee9db4b uri = /config
20932025-04-30T06:43:44.368ZINFOtest_unauthorized (store): attempting generation update new_generation = 2 req_id = b50ea46e-7bb5-4f5d-8e9e-eedafee9db4b
20942025-04-30T06:43:44.368ZINFOtest_unauthorized (store): pruning trees for generations newer than 0
20952025-04-30T06:43:44.368ZDEBGtest_unauthorized (store): creating tree tree_name = generation_2_zone_oxide-dev.test
20962025-04-30T06:43:44.368ZDEBGtest_unauthorized (store): updating current config
20972025-04-30T06:43:44.368ZDEBGtest_unauthorized (store): flushing default tree
20982025-04-30T06:43:44.368ZINFOtest_unauthorized (store): pruning trees for generations older than 2 keep = 3
20992025-04-30T06:43:44.368ZINFOtest_unauthorized (store): updated generation new_generation = 2 req_id = b50ea46e-7bb5-4f5d-8e9e-eedafee9db4b
21002025-04-30T06:43:44.369ZINFOtest_unauthorized (http): request completed kind = dns latency_us = 2723 local_addr = [::1]:61288 method = PUT remote_addr = [::1]:40820 req_id = b50ea46e-7bb5-4f5d-8e9e-eedafee9db4b response_code = 204 uri = /config
21012025-04-30T06:43:44.451ZINFOtest_unauthorized (dropshot): accepted connection kind = pantry local_addr = [::1]:41451 remote_addr = [::1]:47577 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
21022025-04-30T06:43:44.454ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "97cee87e-53af-4b30-a354-f1e838d87da7", "content-length": "835", "date": "Wed, 30 Apr 2025 06:43:43 GMT"} })
21032025-04-30T06:43:44.455ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
21042025-04-30T06:43:44.457ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[::1]:61288 generation = 2 result = Ok(Response { url: "http://[::1]:61288/config", status: 204, headers: {"x-request-id": "b50ea46e-7bb5-4f5d-8e9e-eedafee9db4b", "date": "Wed, 30 Apr 2025 06:43:43 GMT"} }) servers = DnsServersList { addresses: [[::1]:61288] }
21052025-04-30T06:43:44.457ZINFOtest_unauthorized (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[::1]:61288 generation = 2 servers = DnsServersList { addresses: [[::1]:61288] }
21062025-04-30T06:43:44.461ZINFOtest_unauthorized (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external generation = 2 servers = DnsServersList { addresses: [[::1]:61288] }
21072025-04-30T06:43:44.461ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 167.329223ms iteration = 3 status = {"generation":2,"server_results":{"[::1]:61288":{"Ok":null}}}
21082025-04-30T06:43:44.462ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
21092025-04-30T06:43:44.462ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21102025-04-30T06:43:44.462ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
21112025-04-30T06:43:44.464ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21122025-04-30T06:43:44.464ZTRACtest_unauthorized (dropshot): incoming request kind = pantry local_addr = [::1]:41451 method = GET remote_addr = [::1]:47577 req_id = 6a539eb8-0c06-4823-b286-35406abe31ea sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
21132025-04-30T06:43:44.465ZINFOtest_unauthorized (dropshot): request completed kind = pantry latency_us = 1235 local_addr = [::1]:41451 method = GET remote_addr = [::1]:47577 req_id = 6a539eb8-0c06-4823-b286-35406abe31ea response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
21142025-04-30T06:43:45.032ZINFOtest_unauthorized (dropshot): accepted connection kind = pantry local_addr = [::1]:41451 remote_addr = [::1]:61329 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
21152025-04-30T06:43:45.032ZINFOtest_unauthorized (dropshot): accepted connection kind = pantry local_addr = [::1]:41451 remote_addr = [::1]:45122 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
21162025-04-30T06:43:45.032ZINFOtest_unauthorized (dropshot): accepted connection kind = pantry local_addr = [::1]:41451 remote_addr = [::1]:63718 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
21172025-04-30T06:43:45.032ZINFOtest_unauthorized (dropshot): accepted connection kind = pantry local_addr = [::1]:41451 remote_addr = [::1]:57365 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
21182025-04-30T06:43:45.032ZINFOtest_unauthorized (dropshot): accepted connection kind = pantry local_addr = [::1]:41451 remote_addr = [::1]:62374 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
21192025-04-30T06:43:45.032ZINFOtest_unauthorized (dropshot): accepted connection kind = pantry local_addr = [::1]:41451 remote_addr = [::1]:42997 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
21202025-04-30T06:43:45.032ZINFOtest_unauthorized (dropshot): accepted connection kind = pantry local_addr = [::1]:41451 remote_addr = [::1]:59766 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
21212025-04-30T06:43:45.034ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
21222025-04-30T06:43:45.035ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21232025-04-30T06:43:45.035ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
21242025-04-30T06:43:45.036ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21252025-04-30T06:43:45.036ZTRACtest_unauthorized (dropshot): incoming request kind = pantry local_addr = [::1]:41451 method = GET remote_addr = [::1]:61329 req_id = 4a765be6-e432-4a6f-877b-142410c99111 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
21262025-04-30T06:43:45.037ZTRACtest_unauthorized (dropshot): incoming request kind = pantry local_addr = [::1]:41451 method = GET remote_addr = [::1]:45122 req_id = 491c4230-35bb-4917-996c-377d0a2592db sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
21272025-04-30T06:43:45.037ZTRACtest_unauthorized (dropshot): incoming request kind = pantry local_addr = [::1]:41451 method = GET remote_addr = [::1]:63718 req_id = f1af6eda-2a9a-4979-bf88-2854949e7f25 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
21282025-04-30T06:43:45.037ZTRACtest_unauthorized (dropshot): incoming request kind = pantry local_addr = [::1]:41451 method = GET remote_addr = [::1]:57365 req_id = 3d2faeae-9eda-4d73-ad9c-6e2c7570118d sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
21292025-04-30T06:43:45.037ZTRACtest_unauthorized (dropshot): incoming request kind = pantry local_addr = [::1]:41451 method = GET remote_addr = [::1]:62374 req_id = a132783d-5796-4bd5-a41f-b4948ff6b19e sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
21302025-04-30T06:43:45.037ZTRACtest_unauthorized (dropshot): incoming request kind = pantry local_addr = [::1]:41451 method = GET remote_addr = [::1]:42997 req_id = 1ddf3114-133a-44c4-a9f3-6397898e252b sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
21312025-04-30T06:43:45.038ZTRACtest_unauthorized (dropshot): incoming request kind = pantry local_addr = [::1]:41451 method = GET remote_addr = [::1]:59766 req_id = e98dcabd-b71c-4cf0-8a9d-8567a5da6293 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
21322025-04-30T06:43:45.038ZINFOtest_unauthorized (dropshot): request completed kind = pantry latency_us = 1534 local_addr = [::1]:41451 method = GET remote_addr = [::1]:61329 req_id = 4a765be6-e432-4a6f-877b-142410c99111 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
21332025-04-30T06:43:45.038ZINFOtest_unauthorized (dropshot): request completed kind = pantry latency_us = 1505 local_addr = [::1]:41451 method = GET remote_addr = [::1]:45122 req_id = 491c4230-35bb-4917-996c-377d0a2592db response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
21342025-04-30T06:43:45.038ZINFOtest_unauthorized (dropshot): request completed kind = pantry latency_us = 1465 local_addr = [::1]:41451 method = GET remote_addr = [::1]:63718 req_id = f1af6eda-2a9a-4979-bf88-2854949e7f25 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
21352025-04-30T06:43:45.038ZINFOtest_unauthorized (dropshot): request completed kind = pantry latency_us = 1329 local_addr = [::1]:41451 method = GET remote_addr = [::1]:57365 req_id = 3d2faeae-9eda-4d73-ad9c-6e2c7570118d response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
21362025-04-30T06:43:45.038ZINFOtest_unauthorized (dropshot): request completed kind = pantry latency_us = 1246 local_addr = [::1]:41451 method = GET remote_addr = [::1]:62374 req_id = a132783d-5796-4bd5-a41f-b4948ff6b19e response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
21372025-04-30T06:43:45.038ZINFOtest_unauthorized (dropshot): request completed kind = pantry latency_us = 1092 local_addr = [::1]:41451 method = GET remote_addr = [::1]:42997 req_id = 1ddf3114-133a-44c4-a9f3-6397898e252b response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
21382025-04-30T06:43:45.038ZINFOtest_unauthorized (dropshot): request completed kind = pantry latency_us = 1105 local_addr = [::1]:41451 method = GET remote_addr = [::1]:59766 req_id = e98dcabd-b71c-4cf0-8a9d-8567a5da6293 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
21392025-04-30T06:43:45.042ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21402025-04-30T06:43:45.061ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
21412025-04-30T06:43:45.061ZTRACtest_unauthorized (InternalApi): successful lookup of siloed resource "Vpc" using built-in user
21422025-04-30T06:43:45.061ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
21432025-04-30T06:43:45.061ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
21442025-04-30T06:43:45.061ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21452025-04-30T06:43:45.061ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
21462025-04-30T06:43:45.064ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21472025-04-30T06:43:45.070ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "31b662ba-ce15-4fd4-af9d-55f1589d618c", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:45 GMT"} })
21482025-04-30T06:43:45.070ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
21492025-04-30T06:43:45.070ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
21502025-04-30T06:43:45.070ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21512025-04-30T06:43:45.070ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
21522025-04-30T06:43:45.071ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21532025-04-30T06:43:45.083ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
21542025-04-30T06:43:45.084ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21552025-04-30T06:43:45.084ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
21562025-04-30T06:43:45.085ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21572025-04-30T06:43:45.096ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21582025-04-30T06:43:45.109ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
21592025-04-30T06:43:45.109ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21602025-04-30T06:43:45.109ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
21612025-04-30T06:43:45.110ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21622025-04-30T06:43:45.118ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21632025-04-30T06:43:45.118ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
21642025-04-30T06:43:45.119ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21652025-04-30T06:43:45.126ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ac03ca9a-aed9-4f12-8243-3651be085b08", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:45 GMT"} })
21662025-04-30T06:43:45.126ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
21672025-04-30T06:43:45.126ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21682025-04-30T06:43:45.126ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
21692025-04-30T06:43:45.127ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21702025-04-30T06:43:45.130ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21712025-04-30T06:43:45.130ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
21722025-04-30T06:43:45.131ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21732025-04-30T06:43:45.133ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0a9a5dd8-14d5-4a2d-a1e0-0580b30935ed", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:45 GMT"} })
21742025-04-30T06:43:45.133ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
21752025-04-30T06:43:45.135ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
21762025-04-30T06:43:45.135ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
21772025-04-30T06:43:45.135ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21782025-04-30T06:43:45.135ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
21792025-04-30T06:43:45.136ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21802025-04-30T06:43:45.138ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4bc8b359-dfc8-4d2e-b126-97935f9f5725", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:45 GMT"} })
21812025-04-30T06:43:45.138ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
21822025-04-30T06:43:45.138ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
21832025-04-30T06:43:45.138ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21842025-04-30T06:43:45.138ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
21852025-04-30T06:43:45.139ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21862025-04-30T06:43:45.154ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
21872025-04-30T06:43:45.154ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21882025-04-30T06:43:45.154ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
21892025-04-30T06:43:45.155ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21902025-04-30T06:43:45.156ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b282399c-bc4a-46de-b73a-b3aebccbd0c7", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:45 GMT"} })
21912025-04-30T06:43:45.156ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
21922025-04-30T06:43:45.158ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21932025-04-30T06:43:45.166ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
21942025-04-30T06:43:45.166ZTRACtest_unauthorized (InternalApi): successful lookup of siloed resource "Vpc" using built-in user
21952025-04-30T06:43:45.166ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21962025-04-30T06:43:45.166ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
21972025-04-30T06:43:45.166ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21982025-04-30T06:43:45.167ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a045c7f1-ed8b-4936-82f0-9b7e16b03062", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:45 GMT"} })
21992025-04-30T06:43:45.167ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
22002025-04-30T06:43:45.169ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22012025-04-30T06:43:45.169ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
22022025-04-30T06:43:45.169ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22032025-04-30T06:43:45.172ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "66240a0d-a6a6-4054-aa84-eaaa985c784f", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:45 GMT"} })
22042025-04-30T06:43:45.172ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
22052025-04-30T06:43:45.173ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22062025-04-30T06:43:45.173ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
22072025-04-30T06:43:45.174ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22082025-04-30T06:43:45.175ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cf394817-f0fc-4219-bf0e-0aa8c51a13f9", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:45 GMT"} })
22092025-04-30T06:43:45.175ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
22102025-04-30T06:43:45.176ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22112025-04-30T06:43:45.177ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
22122025-04-30T06:43:45.177ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22132025-04-30T06:43:45.178ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "53e99b2b-67a7-4c69-8732-d45016385a3c", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:45 GMT"} })
22142025-04-30T06:43:45.178ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
22152025-04-30T06:43:45.179ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7edfdd36-f7d6-4a3b-9c5a-8881754a2dd6", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:45 GMT"} })
22162025-04-30T06:43:45.180ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
22172025-04-30T06:43:45.180ZTRACtest_unauthorized (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") }
22182025-04-30T06:43:45.180ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
22192025-04-30T06:43:45.180ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22202025-04-30T06:43:45.180ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
22212025-04-30T06:43:45.180ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22222025-04-30T06:43:45.182ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "dee8851b-6ff5-4afa-923a-d051d0276d09", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:45 GMT"} })
22232025-04-30T06:43:45.182ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
22242025-04-30T06:43:45.183ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
22252025-04-30T06:43:45.183ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22262025-04-30T06:43:45.183ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
22272025-04-30T06:43:45.183ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22282025-04-30T06:43:45.185ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5159ab72-8d5e-4599-bd38-8c105739db43", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:45 GMT"} })
22292025-04-30T06:43:45.185ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
22302025-04-30T06:43:45.186ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
22312025-04-30T06:43:45.186ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22322025-04-30T06:43:45.186ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
22332025-04-30T06:43:45.186ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22342025-04-30T06:43:45.189ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22352025-04-30T06:43:45.203ZDEBGtest_unauthorized (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
22362025-04-30T06:43:45.204ZTRACtest_unauthorized (InternalApi): successful lookup of siloed resource "VpcSubnet" using built-in user
22372025-04-30T06:43:45.204ZTRACtest_unauthorized (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") }
22382025-04-30T06:43:45.204ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
22392025-04-30T06:43:45.204ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22402025-04-30T06:43:45.204ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
22412025-04-30T06:43:45.204ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22422025-04-30T06:43:45.205ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "db5e9a85-25e8-42ad-a8c9-c5b2fd1fccc0", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:45 GMT"} })
22432025-04-30T06:43:45.205ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
22442025-04-30T06:43:45.208ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
22452025-04-30T06:43:45.208ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22462025-04-30T06:43:45.208ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
22472025-04-30T06:43:45.208ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22482025-04-30T06:43:45.211ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c49f714f-7f91-48d7-af81-967cd214a7ab", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:45 GMT"} })
22492025-04-30T06:43:45.211ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
22502025-04-30T06:43:45.212ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
22512025-04-30T06:43:45.212ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22522025-04-30T06:43:45.212ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
22532025-04-30T06:43:45.213ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22542025-04-30T06:43:45.216ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22552025-04-30T06:43:45.226ZDEBGtest_unauthorized (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
22562025-04-30T06:43:45.226ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22572025-04-30T06:43:45.226ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
22582025-04-30T06:43:45.226ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22592025-04-30T06:43:45.229ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d9f9a39e-78a9-4b65-8964-bab8d7981143", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:45 GMT"} })
22602025-04-30T06:43:45.229ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
22612025-04-30T06:43:45.236ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22622025-04-30T06:43:45.242ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
22632025-04-30T06:43:45.243ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22642025-04-30T06:43:45.246ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22652025-04-30T06:43:45.246ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
22662025-04-30T06:43:45.249ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22672025-04-30T06:43:45.254ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f27a395a-5752-47ed-8868-d12104dbb738", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:45 GMT"} })
22682025-04-30T06:43:45.254ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
22692025-04-30T06:43:45.256ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22702025-04-30T06:43:45.256ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
22712025-04-30T06:43:45.257ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22722025-04-30T06:43:45.259ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "98427ac4-862e-4c15-aeb9-b682bde6efe3", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:45 GMT"} })
22732025-04-30T06:43:45.259ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
22742025-04-30T06:43:45.259ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22752025-04-30T06:43:45.259ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
22762025-04-30T06:43:45.260ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22772025-04-30T06:43:45.262ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9171a317-49fe-49ee-949d-a9dd3ef0a108", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:45 GMT"} })
22782025-04-30T06:43:45.262ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
22792025-04-30T06:43:45.262ZTRACtest_unauthorized (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") }
22802025-04-30T06:43:45.262ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
22812025-04-30T06:43:45.262ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22822025-04-30T06:43:45.262ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
22832025-04-30T06:43:45.263ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22842025-04-30T06:43:45.264ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2d4786b9-4870-4a97-9825-312f9e4fcb52", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:45 GMT"} })
22852025-04-30T06:43:45.264ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
22862025-04-30T06:43:45.265ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9fc19479-31af-4117-8ff0-c44cedbd0856", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:45 GMT"} })
22872025-04-30T06:43:45.265ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
22882025-04-30T06:43:45.266ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
22892025-04-30T06:43:45.266ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22902025-04-30T06:43:45.266ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
22912025-04-30T06:43:45.266ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22922025-04-30T06:43:45.268ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6da12705-ca46-4a2d-8c7a-27f097033748", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:45 GMT"} })
22932025-04-30T06:43:45.268ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
22942025-04-30T06:43:45.269ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
22952025-04-30T06:43:45.269ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22962025-04-30T06:43:45.269ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
22972025-04-30T06:43:45.270ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22982025-04-30T06:43:45.270ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "82767119-f1d7-4f4e-9f18-3fcd48a31018", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:45 GMT"} })
22992025-04-30T06:43:45.270ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
23002025-04-30T06:43:45.272ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "24325771-8de0-4aa7-b5f4-8880c5442021", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:45 GMT"} })
23012025-04-30T06:43:45.272ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
23022025-04-30T06:43:45.272ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23032025-04-30T06:43:45.279ZDEBGtest_unauthorized (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
23042025-04-30T06:43:45.279ZTRACtest_unauthorized (InternalApi): successful lookup of siloed resource "VpcSubnet" using built-in user
23052025-04-30T06:43:45.279ZTRACtest_unauthorized (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") }
23062025-04-30T06:43:45.279ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
23072025-04-30T06:43:45.279ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23082025-04-30T06:43:45.279ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
23092025-04-30T06:43:45.280ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23102025-04-30T06:43:45.282ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
23112025-04-30T06:43:45.282ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23122025-04-30T06:43:45.283ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
23132025-04-30T06:43:45.283ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23142025-04-30T06:43:45.283ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc8ad1de-2956-4870-b89b-6280fa452ce2", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:45 GMT"} })
23152025-04-30T06:43:45.283ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
23162025-04-30T06:43:45.286ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
23172025-04-30T06:43:45.286ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23182025-04-30T06:43:45.286ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
23192025-04-30T06:43:45.287ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23202025-04-30T06:43:45.288ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "df3278a6-5038-4924-a155-8faa850e312b", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:45 GMT"} })
23212025-04-30T06:43:45.288ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
23222025-04-30T06:43:45.289ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23232025-04-30T06:43:45.298ZDEBGtest_unauthorized (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
23242025-04-30T06:43:45.298ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23252025-04-30T06:43:45.298ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
23262025-04-30T06:43:45.298ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23272025-04-30T06:43:45.301ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cec7f231-e25a-4d70-96a1-12d0fb56b44e", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:45 GMT"} })
23282025-04-30T06:43:45.301ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
23292025-04-30T06:43:45.303ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "267abf83-e5f3-49cb-ad19-1503e96d595b", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:45 GMT"} })
23302025-04-30T06:43:45.303ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
23312025-04-30T06:43:45.305ZDEBGtest_unauthorized (InternalApi): resolved names for firewall rules instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 5036fcc9-7261-43e3-9e27-69fb4ef2e46e, kind: Service { id: 8d752adb-08b9-405f-a11b-66019bf447b5 }, name: Name("nexus-8d752adb-08b9-405f-a11b-66019bf447b5"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 1])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("external-dns"): [NetworkInterface { id: 3466d994-2ae5-4a93-8b25-2ea17242d33b, kind: Service { id: 9a892c5e-6788-4649-8d0f-08ee55eaa359 }, name: Name("external-dns-9a892c5e-6788-4649-8d0f-08ee55eaa359"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 0])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
23322025-04-30T06:43:45.305ZTRACtest_unauthorized (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
23332025-04-30T06:43:45.305ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
23342025-04-30T06:43:45.305ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23352025-04-30T06:43:45.305ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
23362025-04-30T06:43:45.306ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23372025-04-30T06:43:45.308ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6853a80e-e4be-44c8-98ea-a192e06c4108", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:45 GMT"} })
23382025-04-30T06:43:45.308ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
23392025-04-30T06:43:45.309ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23402025-04-30T06:43:45.314ZDEBGtest_unauthorized (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
23412025-04-30T06:43:45.314ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23422025-04-30T06:43:45.314ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
23432025-04-30T06:43:45.314ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23442025-04-30T06:43:45.316ZTRACtest_unauthorized (InternalApi): fetched allowlist from DB allowed = AllowList { id: 001de000-a110-4000-8000-000000000000, time_created: 2025-04-30T06:06:08.248500Z, time_modified: 2025-04-30T06:43:43.226964Z, allowed_ips: None }
23452025-04-30T06:43:45.317ZDEBGtest_unauthorized (InternalApi): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified.
23462025-04-30T06:43:45.317ZDEBGtest_unauthorized (InternalApi): resolved firewall rules for sled agents sled_agent_rules = [ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 3466d994-2ae5-4a93-8b25-2ea17242d33b, kind: Service { id: 9a892c5e-6788-4649-8d0f-08ee55eaa359 }, name: Name("external-dns-9a892c5e-6788-4649-8d0f-08ee55eaa359"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 0])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }, ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 5036fcc9-7261-43e3-9e27-69fb4ef2e46e, kind: Service { id: 8d752adb-08b9-405f-a11b-66019bf447b5 }, name: Name("nexus-8d752adb-08b9-405f-a11b-66019bf447b5"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 1])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }]
23472025-04-30T06:43:45.317ZDEBGtest_unauthorized (InternalApi): resolved 2 rules for sleds
23482025-04-30T06:43:45.320ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b788f141-d932-4478-aeda-1ca11d9828fe", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:45 GMT"} })
23492025-04-30T06:43:45.320ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
23502025-04-30T06:43:45.321ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "de4bb4cc-76a4-4833-b5e9-cd0aac300cfd", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:45 GMT"} })
23512025-04-30T06:43:45.322ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
23522025-04-30T06:43:45.322ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5e0b607f-ecdc-4812-be15-691a7ae61af8", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:45 GMT"} })
23532025-04-30T06:43:45.323ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
23542025-04-30T06:43:45.324ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "18c183cb-231a-4d96-b104-c1c4cee39712", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:45 GMT"} })
23552025-04-30T06:43:45.327ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
23562025-04-30T06:43:45.330ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "080e39e0-b8f3-4993-8aeb-99e4bef2a821", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:45 GMT"} })
23572025-04-30T06:43:45.330ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
23582025-04-30T06:43:45.334ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "41a19166-f72a-4193-82b1-ac61f1f3e383", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:45 GMT"} })
23592025-04-30T06:43:45.334ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
23602025-04-30T06:43:45.338ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "958d1575-bdfe-44f9-8091-709564e49058", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:45 GMT"} })
23612025-04-30T06:43:45.338ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
23622025-04-30T06:43:45.338ZDEBGtest_unauthorized (InternalApi): resolved sleds for vpc oxide-services vpc_to_sled = [Sled { identity: SledIdentity { id: b6d65341-167c-41df-9b5c-41cded99c229, time_created: 2025-04-30T06:43:36.474963Z, time_modified: 2025-04-30T06:43:36.474963Z }, time_deleted: None, rcgen: Generation(Generation(1)), rack_id: c19a698f-c6f9-4a17-ae30-20d711b8f7dc, is_scrimlet: true, serial_number: "sim-b6d65341", part_number: "Unknown", revision: SqlU32(0), usable_hardware_threads: SqlU32(16), usable_physical_ram: ByteCount(ByteCount(34359738368)), reservoir_size: ByteCount(ByteCount(17179869184)), ip: ::1, port: SqlU16(57786), last_used_address: ::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(48692) }]
23632025-04-30T06:43:45.338ZDEBGtest_unauthorized (InternalApi): sending firewall rules to sled agents
23642025-04-30T06:43:45.338ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23652025-04-30T06:43:45.338ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
23662025-04-30T06:43:45.340ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23672025-04-30T06:43:45.343ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Sled { parent: Fleet, key: b6d65341-167c-41df-9b5c-41cded99c229, lookup_type: ById(b6d65341-167c-41df-9b5c-41cded99c229) }
23682025-04-30T06:43:45.343ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
23692025-04-30T06:43:45.343ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23702025-04-30T06:43:45.343ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
23712025-04-30T06:43:45.344ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23722025-04-30T06:43:45.345ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9bcb8776-1a26-4d83-bf92-a5a38df806a4", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:45 GMT"} })
23732025-04-30T06:43:45.345ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
23742025-04-30T06:43:45.347ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23752025-04-30T06:43:45.363ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Sled { parent: Fleet, key: b6d65341-167c-41df-9b5c-41cded99c229, lookup_type: ById(b6d65341-167c-41df-9b5c-41cded99c229) } result = Ok(())
23762025-04-30T06:43:45.420ZDEBGtest_unauthorized (InternalApi): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 body = Some(Body) method = PUT uri = http://[::1]:57786/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
23772025-04-30T06:43:45.421ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:45922 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
23782025-04-30T06:43:45.422ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = PUT remote_addr = [::1]:45922 req_id = 3b9cbe5f-4e3e-4272-8bf7-4e5a306e4559 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
23792025-04-30T06:43:45.423ZINFOtest_unauthorized (dropshot): request completed latency_us = 1038 local_addr = [::1]:57786 method = PUT remote_addr = [::1]:45922 req_id = 3b9cbe5f-4e3e-4272-8bf7-4e5a306e4559 response_code = 204 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
23802025-04-30T06:43:45.423ZDEBGtest_unauthorized (InternalApi): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 result = Ok(Response { url: "http://[::1]:57786/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "3b9cbe5f-4e3e-4272-8bf7-4e5a306e4559", "date": "Wed, 30 Apr 2025 06:43:45 GMT"} })
23812025-04-30T06:43:45.424ZINFOtest_unauthorized (InternalApi): updated firewall rules on 1 sleds
23822025-04-30T06:43:45.424ZINFOtest_unauthorized (nexus): plumbed initial IP allowlist
23832025-04-30T06:43:45.481ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = / versions = all
23842025-04-30T06:43:45.481ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /access versions = all
23852025-04-30T06:43:45.481ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /assets/{path} versions = all
23862025-04-30T06:43:45.481ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /device/auth versions = all
23872025-04-30T06:43:45.481ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /device/confirm versions = all
23882025-04-30T06:43:45.481ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /device/success versions = all
23892025-04-30T06:43:45.481ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /device/token versions = all
23902025-04-30T06:43:45.481ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /device/verify versions = all
23912025-04-30T06:43:45.481ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /experimental/v1/probes versions = all
23922025-04-30T06:43:45.481ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /experimental/v1/probes versions = all
23932025-04-30T06:43:45.481ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = DELETE path = /experimental/v1/probes/{probe} versions = all
23942025-04-30T06:43:45.481ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /experimental/v1/probes/{probe} versions = all
23952025-04-30T06:43:45.481ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /experimental/v1/system/support-bundles versions = all
23962025-04-30T06:43:45.481ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /experimental/v1/system/support-bundles versions = all
23972025-04-30T06:43:45.482ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = DELETE path = /experimental/v1/system/support-bundles/{support_bundle} versions = all
23982025-04-30T06:43:45.482ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /experimental/v1/system/support-bundles/{support_bundle} versions = all
23992025-04-30T06:43:45.482ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /experimental/v1/system/support-bundles/{support_bundle}/download versions = all
24002025-04-30T06:43:45.482ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = HEAD path = /experimental/v1/system/support-bundles/{support_bundle}/download versions = all
24012025-04-30T06:43:45.482ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /experimental/v1/system/support-bundles/{support_bundle}/download/{file} versions = all
24022025-04-30T06:43:45.482ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = HEAD path = /experimental/v1/system/support-bundles/{support_bundle}/download/{file} versions = all
24032025-04-30T06:43:45.482ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /experimental/v1/system/support-bundles/{support_bundle}/index versions = all
24042025-04-30T06:43:45.482ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /images versions = all
24052025-04-30T06:43:45.482ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /login versions = all
24062025-04-30T06:43:45.482ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /login/{silo_name}/local versions = all
24072025-04-30T06:43:45.482ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /login/{silo_name}/saml/{provider_name} versions = all
24082025-04-30T06:43:45.482ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /login/{silo_name}/saml/{provider_name} versions = all
24092025-04-30T06:43:45.482ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /login/{silo_name}/saml/{provider_name}/redirect versions = all
24102025-04-30T06:43:45.482ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /lookup/{path} versions = all
24112025-04-30T06:43:45.482ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /projects/{path} versions = all
24122025-04-30T06:43:45.482ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /projects-new versions = all
24132025-04-30T06:43:45.482ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /settings/{path} versions = all
24142025-04-30T06:43:45.482ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /system/{path} versions = all
24152025-04-30T06:43:45.482ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /utilization versions = all
24162025-04-30T06:43:45.482ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/affinity-groups versions = all
24172025-04-30T06:43:45.482ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/affinity-groups versions = all
24182025-04-30T06:43:45.482ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = DELETE path = /v1/affinity-groups/{affinity_group} versions = all
24192025-04-30T06:43:45.482ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/affinity-groups/{affinity_group} versions = all
24202025-04-30T06:43:45.482ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = PUT path = /v1/affinity-groups/{affinity_group} versions = all
24212025-04-30T06:43:45.482ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/affinity-groups/{affinity_group}/members versions = all
24222025-04-30T06:43:45.482ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = DELETE path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
24232025-04-30T06:43:45.482ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
24242025-04-30T06:43:45.482ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
24252025-04-30T06:43:45.482ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/anti-affinity-groups versions = all
24262025-04-30T06:43:45.482ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/anti-affinity-groups versions = all
24272025-04-30T06:43:45.482ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = DELETE path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
24282025-04-30T06:43:45.482ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
24292025-04-30T06:43:45.482ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = PUT path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
24302025-04-30T06:43:45.482ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group}/members versions = all
24312025-04-30T06:43:45.482ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = DELETE path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
24322025-04-30T06:43:45.482ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
24332025-04-30T06:43:45.482ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
24342025-04-30T06:43:45.482ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/certificates versions = all
24352025-04-30T06:43:45.482ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/certificates versions = all
24362025-04-30T06:43:45.482ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = DELETE path = /v1/certificates/{certificate} versions = all
24372025-04-30T06:43:45.482ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/certificates/{certificate} versions = all
24382025-04-30T06:43:45.482ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/disks versions = all
24392025-04-30T06:43:45.482ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/disks versions = all
24402025-04-30T06:43:45.482ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = DELETE path = /v1/disks/{disk} versions = all
24412025-04-30T06:43:45.482ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/disks/{disk} versions = all
24422025-04-30T06:43:45.483ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/disks/{disk}/bulk-write versions = all
24432025-04-30T06:43:45.483ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/disks/{disk}/bulk-write-start versions = all
24442025-04-30T06:43:45.483ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/disks/{disk}/bulk-write-stop versions = all
24452025-04-30T06:43:45.483ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/disks/{disk}/finalize versions = all
24462025-04-30T06:43:45.483ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/disks/{disk}/metrics/{metric} versions = all
24472025-04-30T06:43:45.483ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/floating-ips versions = all
24482025-04-30T06:43:45.483ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/floating-ips versions = all
24492025-04-30T06:43:45.483ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = DELETE path = /v1/floating-ips/{floating_ip} versions = all
24502025-04-30T06:43:45.483ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/floating-ips/{floating_ip} versions = all
24512025-04-30T06:43:45.483ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = PUT path = /v1/floating-ips/{floating_ip} versions = all
24522025-04-30T06:43:45.483ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/floating-ips/{floating_ip}/attach versions = all
24532025-04-30T06:43:45.483ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/floating-ips/{floating_ip}/detach versions = all
24542025-04-30T06:43:45.483ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/groups versions = all
24552025-04-30T06:43:45.483ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/groups/{group_id} versions = all
24562025-04-30T06:43:45.483ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/images versions = all
24572025-04-30T06:43:45.483ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/images versions = all
24582025-04-30T06:43:45.483ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = DELETE path = /v1/images/{image} versions = all
24592025-04-30T06:43:45.483ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/images/{image} versions = all
24602025-04-30T06:43:45.483ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/images/{image}/demote versions = all
24612025-04-30T06:43:45.483ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/images/{image}/promote versions = all
24622025-04-30T06:43:45.483ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/instances versions = all
24632025-04-30T06:43:45.483ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/instances versions = all
24642025-04-30T06:43:45.483ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = DELETE path = /v1/instances/{instance} versions = all
24652025-04-30T06:43:45.483ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/instances/{instance} versions = all
24662025-04-30T06:43:45.483ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = PUT path = /v1/instances/{instance} versions = all
24672025-04-30T06:43:45.483ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/instances/{instance}/affinity-groups versions = all
24682025-04-30T06:43:45.483ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/instances/{instance}/anti-affinity-groups versions = all
24692025-04-30T06:43:45.483ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/instances/{instance}/disks versions = all
24702025-04-30T06:43:45.483ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/instances/{instance}/disks/attach versions = all
24712025-04-30T06:43:45.483ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/instances/{instance}/disks/detach versions = all
24722025-04-30T06:43:45.483ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/instances/{instance}/external-ips versions = all
24732025-04-30T06:43:45.483ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = DELETE path = /v1/instances/{instance}/external-ips/ephemeral versions = all
24742025-04-30T06:43:45.483ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/instances/{instance}/external-ips/ephemeral versions = all
24752025-04-30T06:43:45.483ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/instances/{instance}/reboot versions = all
24762025-04-30T06:43:45.483ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/instances/{instance}/serial-console versions = all
24772025-04-30T06:43:45.483ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/instances/{instance}/serial-console/stream versions = all
24782025-04-30T06:43:45.483ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/instances/{instance}/ssh-public-keys versions = all
24792025-04-30T06:43:45.483ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/instances/{instance}/start versions = all
24802025-04-30T06:43:45.483ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/instances/{instance}/stop versions = all
24812025-04-30T06:43:45.483ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/internet-gateway-ip-addresses versions = all
24822025-04-30T06:43:45.483ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/internet-gateway-ip-addresses versions = all
24832025-04-30T06:43:45.483ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = DELETE path = /v1/internet-gateway-ip-addresses/{address} versions = all
24842025-04-30T06:43:45.483ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/internet-gateway-ip-pools versions = all
24852025-04-30T06:43:45.483ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/internet-gateway-ip-pools versions = all
24862025-04-30T06:43:45.484ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = DELETE path = /v1/internet-gateway-ip-pools/{pool} versions = all
24872025-04-30T06:43:45.484ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/internet-gateways versions = all
24882025-04-30T06:43:45.484ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/internet-gateways versions = all
24892025-04-30T06:43:45.484ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = DELETE path = /v1/internet-gateways/{gateway} versions = all
24902025-04-30T06:43:45.484ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/internet-gateways/{gateway} versions = all
24912025-04-30T06:43:45.484ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/ip-pools versions = all
24922025-04-30T06:43:45.484ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/ip-pools/{pool} versions = all
24932025-04-30T06:43:45.484ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/login/{silo_name}/local versions = all
24942025-04-30T06:43:45.484ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/logout versions = all
24952025-04-30T06:43:45.484ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/me versions = all
24962025-04-30T06:43:45.484ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/me/groups versions = all
24972025-04-30T06:43:45.484ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/me/ssh-keys versions = all
24982025-04-30T06:43:45.484ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/me/ssh-keys versions = all
24992025-04-30T06:43:45.484ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = DELETE path = /v1/me/ssh-keys/{ssh_key} versions = all
25002025-04-30T06:43:45.484ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/me/ssh-keys/{ssh_key} versions = all
25012025-04-30T06:43:45.484ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/metrics/{metric_name} versions = all
25022025-04-30T06:43:45.484ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/network-interfaces versions = all
25032025-04-30T06:43:45.484ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/network-interfaces versions = all
25042025-04-30T06:43:45.484ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = DELETE path = /v1/network-interfaces/{interface} versions = all
25052025-04-30T06:43:45.484ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/network-interfaces/{interface} versions = all
25062025-04-30T06:43:45.484ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = PUT path = /v1/network-interfaces/{interface} versions = all
25072025-04-30T06:43:45.484ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/ping versions = all
25082025-04-30T06:43:45.484ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/policy versions = all
25092025-04-30T06:43:45.484ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = PUT path = /v1/policy versions = all
25102025-04-30T06:43:45.484ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/projects versions = all
25112025-04-30T06:43:45.484ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/projects versions = all
25122025-04-30T06:43:45.484ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = DELETE path = /v1/projects/{project} versions = all
25132025-04-30T06:43:45.484ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/projects/{project} versions = all
25142025-04-30T06:43:45.484ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = PUT path = /v1/projects/{project} versions = all
25152025-04-30T06:43:45.484ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/projects/{project}/policy versions = all
25162025-04-30T06:43:45.484ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = PUT path = /v1/projects/{project}/policy versions = all
25172025-04-30T06:43:45.484ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/snapshots versions = all
25182025-04-30T06:43:45.484ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/snapshots versions = all
25192025-04-30T06:43:45.484ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = DELETE path = /v1/snapshots/{snapshot} versions = all
25202025-04-30T06:43:45.484ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/snapshots/{snapshot} versions = all
25212025-04-30T06:43:45.484ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/hardware/disks versions = all
25222025-04-30T06:43:45.484ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/hardware/disks/{disk_id} versions = all
25232025-04-30T06:43:45.484ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/hardware/rack-switch-port/{rack_id}/{switch_location}/{port}/lldp/neighbors versions = all
25242025-04-30T06:43:45.484ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/hardware/racks versions = all
25252025-04-30T06:43:45.484ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/hardware/racks/{rack_id} versions = all
25262025-04-30T06:43:45.484ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/hardware/sleds versions = all
25272025-04-30T06:43:45.484ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/system/hardware/sleds versions = all
25282025-04-30T06:43:45.484ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/hardware/sleds/{sled_id} versions = all
25292025-04-30T06:43:45.484ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/hardware/sleds/{sled_id}/disks versions = all
25302025-04-30T06:43:45.485ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/hardware/sleds/{sled_id}/instances versions = all
25312025-04-30T06:43:45.485ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = PUT path = /v1/system/hardware/sleds/{sled_id}/provision-policy versions = all
25322025-04-30T06:43:45.485ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/hardware/sleds-uninitialized versions = all
25332025-04-30T06:43:45.485ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/hardware/switch-port versions = all
25342025-04-30T06:43:45.485ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/hardware/switch-port/{port}/lldp/config versions = all
25352025-04-30T06:43:45.485ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/system/hardware/switch-port/{port}/lldp/config versions = all
25362025-04-30T06:43:45.485ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = DELETE path = /v1/system/hardware/switch-port/{port}/settings versions = all
25372025-04-30T06:43:45.485ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/system/hardware/switch-port/{port}/settings versions = all
25382025-04-30T06:43:45.485ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/hardware/switch-port/{port}/status versions = all
25392025-04-30T06:43:45.485ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/hardware/switches versions = all
25402025-04-30T06:43:45.485ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/hardware/switches/{switch_id} versions = all
25412025-04-30T06:43:45.485ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/identity-providers versions = all
25422025-04-30T06:43:45.485ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/system/identity-providers/local/users versions = all
25432025-04-30T06:43:45.485ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = DELETE path = /v1/system/identity-providers/local/users/{user_id} versions = all
25442025-04-30T06:43:45.485ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/system/identity-providers/local/users/{user_id}/set-password versions = all
25452025-04-30T06:43:45.485ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/system/identity-providers/saml versions = all
25462025-04-30T06:43:45.485ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/identity-providers/saml/{provider} versions = all
25472025-04-30T06:43:45.485ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/ip-pools versions = all
25482025-04-30T06:43:45.485ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/system/ip-pools versions = all
25492025-04-30T06:43:45.485ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = DELETE path = /v1/system/ip-pools/{pool} versions = all
25502025-04-30T06:43:45.485ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/ip-pools/{pool} versions = all
25512025-04-30T06:43:45.485ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = PUT path = /v1/system/ip-pools/{pool} versions = all
25522025-04-30T06:43:45.485ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/ip-pools/{pool}/ranges versions = all
25532025-04-30T06:43:45.485ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/system/ip-pools/{pool}/ranges/add versions = all
25542025-04-30T06:43:45.485ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/system/ip-pools/{pool}/ranges/remove versions = all
25552025-04-30T06:43:45.485ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/ip-pools/{pool}/silos versions = all
25562025-04-30T06:43:45.485ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/system/ip-pools/{pool}/silos versions = all
25572025-04-30T06:43:45.485ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = DELETE path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
25582025-04-30T06:43:45.485ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = PUT path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
25592025-04-30T06:43:45.485ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/ip-pools/{pool}/utilization versions = all
25602025-04-30T06:43:45.485ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/ip-pools-service versions = all
25612025-04-30T06:43:45.485ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/ip-pools-service/ranges versions = all
25622025-04-30T06:43:45.485ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/system/ip-pools-service/ranges/add versions = all
25632025-04-30T06:43:45.485ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/system/ip-pools-service/ranges/remove versions = all
25642025-04-30T06:43:45.485ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/metrics/{metric_name} versions = all
25652025-04-30T06:43:45.485ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/networking/address-lot versions = all
25662025-04-30T06:43:45.485ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/system/networking/address-lot versions = all
25672025-04-30T06:43:45.485ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = DELETE path = /v1/system/networking/address-lot/{address_lot} versions = all
25682025-04-30T06:43:45.485ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/networking/address-lot/{address_lot}/blocks versions = all
25692025-04-30T06:43:45.485ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/networking/allow-list versions = all
25702025-04-30T06:43:45.485ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = PUT path = /v1/system/networking/allow-list versions = all
25712025-04-30T06:43:45.485ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/system/networking/bfd-disable versions = all
25722025-04-30T06:43:45.485ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/system/networking/bfd-enable versions = all
25732025-04-30T06:43:45.485ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/networking/bfd-status versions = all
25742025-04-30T06:43:45.486ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = DELETE path = /v1/system/networking/bgp versions = all
25752025-04-30T06:43:45.486ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/networking/bgp versions = all
25762025-04-30T06:43:45.486ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/system/networking/bgp versions = all
25772025-04-30T06:43:45.486ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/networking/bgp-announce-set versions = all
25782025-04-30T06:43:45.486ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = PUT path = /v1/system/networking/bgp-announce-set versions = all
25792025-04-30T06:43:45.486ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = DELETE path = /v1/system/networking/bgp-announce-set/{announce_set} versions = all
25802025-04-30T06:43:45.486ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/networking/bgp-announce-set/{announce_set}/announcement versions = all
25812025-04-30T06:43:45.486ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/networking/bgp-exported versions = all
25822025-04-30T06:43:45.486ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/networking/bgp-message-history versions = all
25832025-04-30T06:43:45.486ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/networking/bgp-routes-ipv4 versions = all
25842025-04-30T06:43:45.486ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/networking/bgp-status versions = all
25852025-04-30T06:43:45.486ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/networking/loopback-address versions = all
25862025-04-30T06:43:45.486ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/system/networking/loopback-address versions = all
25872025-04-30T06:43:45.486ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = DELETE path = /v1/system/networking/loopback-address/{rack_id}/{switch_location}/{address}/{subnet_mask} versions = all
25882025-04-30T06:43:45.486ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = DELETE path = /v1/system/networking/switch-port-settings versions = all
25892025-04-30T06:43:45.486ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/networking/switch-port-settings versions = all
25902025-04-30T06:43:45.486ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/system/networking/switch-port-settings versions = all
25912025-04-30T06:43:45.486ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/networking/switch-port-settings/{port} versions = all
25922025-04-30T06:43:45.486ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/policy versions = all
25932025-04-30T06:43:45.486ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = PUT path = /v1/system/policy versions = all
25942025-04-30T06:43:45.486ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/roles versions = all
25952025-04-30T06:43:45.486ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/roles/{role_name} versions = all
25962025-04-30T06:43:45.486ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/silo-quotas versions = all
25972025-04-30T06:43:45.486ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/silos versions = all
25982025-04-30T06:43:45.486ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/system/silos versions = all
25992025-04-30T06:43:45.486ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = DELETE path = /v1/system/silos/{silo} versions = all
26002025-04-30T06:43:45.486ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/silos/{silo} versions = all
26012025-04-30T06:43:45.486ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/silos/{silo}/ip-pools versions = all
26022025-04-30T06:43:45.486ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/silos/{silo}/policy versions = all
26032025-04-30T06:43:45.486ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = PUT path = /v1/system/silos/{silo}/policy versions = all
26042025-04-30T06:43:45.486ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/silos/{silo}/quotas versions = all
26052025-04-30T06:43:45.486ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = PUT path = /v1/system/silos/{silo}/quotas versions = all
26062025-04-30T06:43:45.486ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/system/timeseries/query versions = all
26072025-04-30T06:43:45.486ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/timeseries/schemas versions = all
26082025-04-30T06:43:45.486ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = PUT path = /v1/system/update/repository versions = all
26092025-04-30T06:43:45.486ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/update/repository/{system_version} versions = all
26102025-04-30T06:43:45.486ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/update/target-release versions = all
26112025-04-30T06:43:45.486ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = PUT path = /v1/system/update/target-release versions = all
26122025-04-30T06:43:45.486ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/users versions = all
26132025-04-30T06:43:45.486ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/users/{user_id} versions = all
26142025-04-30T06:43:45.486ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/users-builtin versions = all
26152025-04-30T06:43:45.486ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/users-builtin/{user} versions = all
26162025-04-30T06:43:45.486ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/utilization/silos versions = all
26172025-04-30T06:43:45.487ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/system/utilization/silos/{silo} versions = all
26182025-04-30T06:43:45.487ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/timeseries/query versions = all
26192025-04-30T06:43:45.487ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/users versions = all
26202025-04-30T06:43:45.487ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/utilization versions = all
26212025-04-30T06:43:45.487ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/vpc-firewall-rules versions = all
26222025-04-30T06:43:45.487ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = PUT path = /v1/vpc-firewall-rules versions = all
26232025-04-30T06:43:45.487ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/vpc-router-routes versions = all
26242025-04-30T06:43:45.487ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/vpc-router-routes versions = all
26252025-04-30T06:43:45.487ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = DELETE path = /v1/vpc-router-routes/{route} versions = all
26262025-04-30T06:43:45.487ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/vpc-router-routes/{route} versions = all
26272025-04-30T06:43:45.487ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = PUT path = /v1/vpc-router-routes/{route} versions = all
26282025-04-30T06:43:45.487ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/vpc-routers versions = all
26292025-04-30T06:43:45.487ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/vpc-routers versions = all
26302025-04-30T06:43:45.487ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = DELETE path = /v1/vpc-routers/{router} versions = all
26312025-04-30T06:43:45.487ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/vpc-routers/{router} versions = all
26322025-04-30T06:43:45.487ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = PUT path = /v1/vpc-routers/{router} versions = all
26332025-04-30T06:43:45.487ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/vpc-subnets versions = all
26342025-04-30T06:43:45.487ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/vpc-subnets versions = all
26352025-04-30T06:43:45.487ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = DELETE path = /v1/vpc-subnets/{subnet} versions = all
26362025-04-30T06:43:45.487ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/vpc-subnets/{subnet} versions = all
26372025-04-30T06:43:45.487ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = PUT path = /v1/vpc-subnets/{subnet} versions = all
26382025-04-30T06:43:45.487ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/vpc-subnets/{subnet}/network-interfaces versions = all
26392025-04-30T06:43:45.487ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/vpcs versions = all
26402025-04-30T06:43:45.487ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/vpcs versions = all
26412025-04-30T06:43:45.487ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = DELETE path = /v1/vpcs/{vpc} versions = all
26422025-04-30T06:43:45.487ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/vpcs/{vpc} versions = all
26432025-04-30T06:43:45.487ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = PUT path = /v1/vpcs/{vpc} versions = all
26442025-04-30T06:43:45.487ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/webhooks/deliveries versions = all
26452025-04-30T06:43:45.487ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/webhooks/deliveries/{event_id}/resend versions = all
26462025-04-30T06:43:45.487ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/webhooks/event-classes versions = all
26472025-04-30T06:43:45.487ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/webhooks/receivers versions = all
26482025-04-30T06:43:45.487ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/webhooks/receivers versions = all
26492025-04-30T06:43:45.487ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = DELETE path = /v1/webhooks/receivers/{receiver} versions = all
26502025-04-30T06:43:45.487ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/webhooks/receivers/{receiver} versions = all
26512025-04-30T06:43:45.487ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = PUT path = /v1/webhooks/receivers/{receiver} versions = all
26522025-04-30T06:43:45.487ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/webhooks/receivers/{receiver}/probe versions = all
26532025-04-30T06:43:45.487ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/webhooks/receivers/{receiver}/subscriptions versions = all
26542025-04-30T06:43:45.487ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = DELETE path = /v1/webhooks/receivers/{receiver}/subscriptions/{subscription} versions = all
26552025-04-30T06:43:45.487ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = GET path = /v1/webhooks/secrets versions = all
26562025-04-30T06:43:45.487ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = POST path = /v1/webhooks/secrets versions = all
26572025-04-30T06:43:45.487ZDEBGtest_unauthorized (dropshot_external): registered endpoint local_addr = 127.0.0.1:57120 method = DELETE path = /v1/webhooks/secrets/{secret_id} versions = all
26582025-04-30T06:43:45.487ZINFOtest_unauthorized (dropshot_external): listening local_addr = 127.0.0.1:57120
26592025-04-30T06:43:45.487ZDEBGtest_unauthorized (dropshot_external): successfully registered DTrace USDT probes local_addr = 127.0.0.1:57120
26602025-04-30T06:43:45.560ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = / versions = all
26612025-04-30T06:43:45.560ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /access versions = all
26622025-04-30T06:43:45.560ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /assets/{path} versions = all
26632025-04-30T06:43:45.560ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /device/auth versions = all
26642025-04-30T06:43:45.560ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /device/confirm versions = all
26652025-04-30T06:43:45.560ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /device/success versions = all
26662025-04-30T06:43:45.560ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /device/token versions = all
26672025-04-30T06:43:45.560ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /device/verify versions = all
26682025-04-30T06:43:45.560ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /experimental/v1/probes versions = all
26692025-04-30T06:43:45.561ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /experimental/v1/probes versions = all
26702025-04-30T06:43:45.561ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = DELETE path = /experimental/v1/probes/{probe} versions = all
26712025-04-30T06:43:45.561ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /experimental/v1/probes/{probe} versions = all
26722025-04-30T06:43:45.561ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /experimental/v1/system/support-bundles versions = all
26732025-04-30T06:43:45.561ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /experimental/v1/system/support-bundles versions = all
26742025-04-30T06:43:45.561ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = DELETE path = /experimental/v1/system/support-bundles/{support_bundle} versions = all
26752025-04-30T06:43:45.561ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /experimental/v1/system/support-bundles/{support_bundle} versions = all
26762025-04-30T06:43:45.561ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /experimental/v1/system/support-bundles/{support_bundle}/download versions = all
26772025-04-30T06:43:45.561ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = HEAD path = /experimental/v1/system/support-bundles/{support_bundle}/download versions = all
26782025-04-30T06:43:45.562ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /experimental/v1/system/support-bundles/{support_bundle}/download/{file} versions = all
26792025-04-30T06:43:45.562ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = HEAD path = /experimental/v1/system/support-bundles/{support_bundle}/download/{file} versions = all
26802025-04-30T06:43:45.562ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /experimental/v1/system/support-bundles/{support_bundle}/index versions = all
26812025-04-30T06:43:45.562ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /images versions = all
26822025-04-30T06:43:45.562ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /login versions = all
26832025-04-30T06:43:45.562ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /login/{silo_name}/local versions = all
26842025-04-30T06:43:45.562ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /login/{silo_name}/saml/{provider_name} versions = all
26852025-04-30T06:43:45.562ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /login/{silo_name}/saml/{provider_name} versions = all
26862025-04-30T06:43:45.562ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /login/{silo_name}/saml/{provider_name}/redirect versions = all
26872025-04-30T06:43:45.562ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /lookup/{path} versions = all
26882025-04-30T06:43:45.563ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /projects/{path} versions = all
26892025-04-30T06:43:45.563ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /projects-new versions = all
26902025-04-30T06:43:45.563ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /settings/{path} versions = all
26912025-04-30T06:43:45.563ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /system/{path} versions = all
26922025-04-30T06:43:45.563ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /utilization versions = all
26932025-04-30T06:43:45.563ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/affinity-groups versions = all
26942025-04-30T06:43:45.563ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/affinity-groups versions = all
26952025-04-30T06:43:45.563ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = DELETE path = /v1/affinity-groups/{affinity_group} versions = all
26962025-04-30T06:43:45.563ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/affinity-groups/{affinity_group} versions = all
26972025-04-30T06:43:45.563ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = PUT path = /v1/affinity-groups/{affinity_group} versions = all
26982025-04-30T06:43:45.563ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/affinity-groups/{affinity_group}/members versions = all
26992025-04-30T06:43:45.563ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = DELETE path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
27002025-04-30T06:43:45.564ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
27012025-04-30T06:43:45.564ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
27022025-04-30T06:43:45.564ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/anti-affinity-groups versions = all
27032025-04-30T06:43:45.564ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/anti-affinity-groups versions = all
27042025-04-30T06:43:45.564ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = DELETE path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
27052025-04-30T06:43:45.564ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
27062025-04-30T06:43:45.564ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = PUT path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
27072025-04-30T06:43:45.564ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group}/members versions = all
27082025-04-30T06:43:45.564ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = DELETE path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
27092025-04-30T06:43:45.564ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
27102025-04-30T06:43:45.564ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
27112025-04-30T06:43:45.564ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/certificates versions = all
27122025-04-30T06:43:45.565ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/certificates versions = all
27132025-04-30T06:43:45.565ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = DELETE path = /v1/certificates/{certificate} versions = all
27142025-04-30T06:43:45.565ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/certificates/{certificate} versions = all
27152025-04-30T06:43:45.565ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/disks versions = all
27162025-04-30T06:43:45.565ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/disks versions = all
27172025-04-30T06:43:45.565ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = DELETE path = /v1/disks/{disk} versions = all
27182025-04-30T06:43:45.565ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/disks/{disk} versions = all
27192025-04-30T06:43:45.565ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/disks/{disk}/bulk-write versions = all
27202025-04-30T06:43:45.566ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/disks/{disk}/bulk-write-start versions = all
27212025-04-30T06:43:45.566ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/disks/{disk}/bulk-write-stop versions = all
27222025-04-30T06:43:45.566ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/disks/{disk}/finalize versions = all
27232025-04-30T06:43:45.566ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/disks/{disk}/metrics/{metric} versions = all
27242025-04-30T06:43:45.566ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/floating-ips versions = all
27252025-04-30T06:43:45.566ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/floating-ips versions = all
27262025-04-30T06:43:45.566ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = DELETE path = /v1/floating-ips/{floating_ip} versions = all
27272025-04-30T06:43:45.566ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/floating-ips/{floating_ip} versions = all
27282025-04-30T06:43:45.566ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = PUT path = /v1/floating-ips/{floating_ip} versions = all
27292025-04-30T06:43:45.567ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/floating-ips/{floating_ip}/attach versions = all
27302025-04-30T06:43:45.567ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/floating-ips/{floating_ip}/detach versions = all
27312025-04-30T06:43:45.567ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/groups versions = all
27322025-04-30T06:43:45.567ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/groups/{group_id} versions = all
27332025-04-30T06:43:45.567ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/images versions = all
27342025-04-30T06:43:45.567ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/images versions = all
27352025-04-30T06:43:45.567ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = DELETE path = /v1/images/{image} versions = all
27362025-04-30T06:43:45.567ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/images/{image} versions = all
27372025-04-30T06:43:45.567ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/images/{image}/demote versions = all
27382025-04-30T06:43:45.568ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/images/{image}/promote versions = all
27392025-04-30T06:43:45.568ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/instances versions = all
27402025-04-30T06:43:45.568ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/instances versions = all
27412025-04-30T06:43:45.568ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = DELETE path = /v1/instances/{instance} versions = all
27422025-04-30T06:43:45.568ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/instances/{instance} versions = all
27432025-04-30T06:43:45.568ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = PUT path = /v1/instances/{instance} versions = all
27442025-04-30T06:43:45.568ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/instances/{instance}/affinity-groups versions = all
27452025-04-30T06:43:45.568ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/instances/{instance}/anti-affinity-groups versions = all
27462025-04-30T06:43:45.568ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/instances/{instance}/disks versions = all
27472025-04-30T06:43:45.568ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/instances/{instance}/disks/attach versions = all
27482025-04-30T06:43:45.568ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/instances/{instance}/disks/detach versions = all
27492025-04-30T06:43:45.568ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/instances/{instance}/external-ips versions = all
27502025-04-30T06:43:45.569ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = DELETE path = /v1/instances/{instance}/external-ips/ephemeral versions = all
27512025-04-30T06:43:45.569ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/instances/{instance}/external-ips/ephemeral versions = all
27522025-04-30T06:43:45.569ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/instances/{instance}/reboot versions = all
27532025-04-30T06:43:45.569ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/instances/{instance}/serial-console versions = all
27542025-04-30T06:43:45.569ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/instances/{instance}/serial-console/stream versions = all
27552025-04-30T06:43:45.569ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/instances/{instance}/ssh-public-keys versions = all
27562025-04-30T06:43:45.569ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/instances/{instance}/start versions = all
27572025-04-30T06:43:45.569ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/instances/{instance}/stop versions = all
27582025-04-30T06:43:45.569ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/internet-gateway-ip-addresses versions = all
27592025-04-30T06:43:45.570ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/internet-gateway-ip-addresses versions = all
27602025-04-30T06:43:45.570ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = DELETE path = /v1/internet-gateway-ip-addresses/{address} versions = all
27612025-04-30T06:43:45.570ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/internet-gateway-ip-pools versions = all
27622025-04-30T06:43:45.570ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/internet-gateway-ip-pools versions = all
27632025-04-30T06:43:45.570ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = DELETE path = /v1/internet-gateway-ip-pools/{pool} versions = all
27642025-04-30T06:43:45.570ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/internet-gateways versions = all
27652025-04-30T06:43:45.570ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/internet-gateways versions = all
27662025-04-30T06:43:45.570ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = DELETE path = /v1/internet-gateways/{gateway} versions = all
27672025-04-30T06:43:45.570ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/internet-gateways/{gateway} versions = all
27682025-04-30T06:43:45.570ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/ip-pools versions = all
27692025-04-30T06:43:45.570ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/ip-pools/{pool} versions = all
27702025-04-30T06:43:45.570ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/login/{silo_name}/local versions = all
27712025-04-30T06:43:45.571ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/logout versions = all
27722025-04-30T06:43:45.571ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/me versions = all
27732025-04-30T06:43:45.571ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/me/groups versions = all
27742025-04-30T06:43:45.571ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/me/ssh-keys versions = all
27752025-04-30T06:43:45.571ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/me/ssh-keys versions = all
27762025-04-30T06:43:45.571ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = DELETE path = /v1/me/ssh-keys/{ssh_key} versions = all
27772025-04-30T06:43:45.572ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/me/ssh-keys/{ssh_key} versions = all
27782025-04-30T06:43:45.572ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/metrics/{metric_name} versions = all
27792025-04-30T06:43:45.572ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/network-interfaces versions = all
27802025-04-30T06:43:45.572ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/network-interfaces versions = all
27812025-04-30T06:43:45.572ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = DELETE path = /v1/network-interfaces/{interface} versions = all
27822025-04-30T06:43:45.572ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/network-interfaces/{interface} versions = all
27832025-04-30T06:43:45.572ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = PUT path = /v1/network-interfaces/{interface} versions = all
27842025-04-30T06:43:45.572ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/ping versions = all
27852025-04-30T06:43:45.572ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/policy versions = all
27862025-04-30T06:43:45.572ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = PUT path = /v1/policy versions = all
27872025-04-30T06:43:45.572ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/projects versions = all
27882025-04-30T06:43:45.573ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/projects versions = all
27892025-04-30T06:43:45.573ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = DELETE path = /v1/projects/{project} versions = all
27902025-04-30T06:43:45.573ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/projects/{project} versions = all
27912025-04-30T06:43:45.573ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = PUT path = /v1/projects/{project} versions = all
27922025-04-30T06:43:45.573ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/projects/{project}/policy versions = all
27932025-04-30T06:43:45.573ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = PUT path = /v1/projects/{project}/policy versions = all
27942025-04-30T06:43:45.573ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/snapshots versions = all
27952025-04-30T06:43:45.573ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/snapshots versions = all
27962025-04-30T06:43:45.573ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = DELETE path = /v1/snapshots/{snapshot} versions = all
27972025-04-30T06:43:45.573ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/snapshots/{snapshot} versions = all
27982025-04-30T06:43:45.573ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/hardware/disks versions = all
27992025-04-30T06:43:45.573ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/hardware/disks/{disk_id} versions = all
28002025-04-30T06:43:45.573ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/hardware/rack-switch-port/{rack_id}/{switch_location}/{port}/lldp/neighbors versions = all
28012025-04-30T06:43:45.574ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/hardware/racks versions = all
28022025-04-30T06:43:45.574ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/hardware/racks/{rack_id} versions = all
28032025-04-30T06:43:45.574ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/hardware/sleds versions = all
28042025-04-30T06:43:45.574ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/system/hardware/sleds versions = all
28052025-04-30T06:43:45.574ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/hardware/sleds/{sled_id} versions = all
28062025-04-30T06:43:45.574ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/hardware/sleds/{sled_id}/disks versions = all
28072025-04-30T06:43:45.574ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/hardware/sleds/{sled_id}/instances versions = all
28082025-04-30T06:43:45.574ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = PUT path = /v1/system/hardware/sleds/{sled_id}/provision-policy versions = all
28092025-04-30T06:43:45.574ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/hardware/sleds-uninitialized versions = all
28102025-04-30T06:43:45.574ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/hardware/switch-port versions = all
28112025-04-30T06:43:45.574ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/hardware/switch-port/{port}/lldp/config versions = all
28122025-04-30T06:43:45.575ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/system/hardware/switch-port/{port}/lldp/config versions = all
28132025-04-30T06:43:45.575ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = DELETE path = /v1/system/hardware/switch-port/{port}/settings versions = all
28142025-04-30T06:43:45.575ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/system/hardware/switch-port/{port}/settings versions = all
28152025-04-30T06:43:45.575ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/hardware/switch-port/{port}/status versions = all
28162025-04-30T06:43:45.575ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/hardware/switches versions = all
28172025-04-30T06:43:45.575ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/hardware/switches/{switch_id} versions = all
28182025-04-30T06:43:45.575ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/identity-providers versions = all
28192025-04-30T06:43:45.575ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/system/identity-providers/local/users versions = all
28202025-04-30T06:43:45.575ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = DELETE path = /v1/system/identity-providers/local/users/{user_id} versions = all
28212025-04-30T06:43:45.575ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/system/identity-providers/local/users/{user_id}/set-password versions = all
28222025-04-30T06:43:45.575ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/system/identity-providers/saml versions = all
28232025-04-30T06:43:45.576ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/identity-providers/saml/{provider} versions = all
28242025-04-30T06:43:45.576ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/ip-pools versions = all
28252025-04-30T06:43:45.576ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/system/ip-pools versions = all
28262025-04-30T06:43:45.576ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = DELETE path = /v1/system/ip-pools/{pool} versions = all
28272025-04-30T06:43:45.576ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/ip-pools/{pool} versions = all
28282025-04-30T06:43:45.576ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = PUT path = /v1/system/ip-pools/{pool} versions = all
28292025-04-30T06:43:45.576ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/ip-pools/{pool}/ranges versions = all
28302025-04-30T06:43:45.576ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/system/ip-pools/{pool}/ranges/add versions = all
28312025-04-30T06:43:45.576ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/system/ip-pools/{pool}/ranges/remove versions = all
28322025-04-30T06:43:45.577ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/ip-pools/{pool}/silos versions = all
28332025-04-30T06:43:45.577ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/system/ip-pools/{pool}/silos versions = all
28342025-04-30T06:43:45.577ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = DELETE path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
28352025-04-30T06:43:45.577ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = PUT path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
28362025-04-30T06:43:45.577ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/ip-pools/{pool}/utilization versions = all
28372025-04-30T06:43:45.577ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/ip-pools-service versions = all
28382025-04-30T06:43:45.577ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/ip-pools-service/ranges versions = all
28392025-04-30T06:43:45.577ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/system/ip-pools-service/ranges/add versions = all
28402025-04-30T06:43:45.577ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/system/ip-pools-service/ranges/remove versions = all
28412025-04-30T06:43:45.578ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/metrics/{metric_name} versions = all
28422025-04-30T06:43:45.578ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/networking/address-lot versions = all
28432025-04-30T06:43:45.578ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/system/networking/address-lot versions = all
28442025-04-30T06:43:45.578ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = DELETE path = /v1/system/networking/address-lot/{address_lot} versions = all
28452025-04-30T06:43:45.578ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/networking/address-lot/{address_lot}/blocks versions = all
28462025-04-30T06:43:45.578ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/networking/allow-list versions = all
28472025-04-30T06:43:45.578ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = PUT path = /v1/system/networking/allow-list versions = all
28482025-04-30T06:43:45.578ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/system/networking/bfd-disable versions = all
28492025-04-30T06:43:45.578ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/system/networking/bfd-enable versions = all
28502025-04-30T06:43:45.578ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/networking/bfd-status versions = all
28512025-04-30T06:43:45.578ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = DELETE path = /v1/system/networking/bgp versions = all
28522025-04-30T06:43:45.579ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/networking/bgp versions = all
28532025-04-30T06:43:45.579ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/system/networking/bgp versions = all
28542025-04-30T06:43:45.579ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/networking/bgp-announce-set versions = all
28552025-04-30T06:43:45.579ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = PUT path = /v1/system/networking/bgp-announce-set versions = all
28562025-04-30T06:43:45.579ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = DELETE path = /v1/system/networking/bgp-announce-set/{announce_set} versions = all
28572025-04-30T06:43:45.579ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/networking/bgp-announce-set/{announce_set}/announcement versions = all
28582025-04-30T06:43:45.579ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/networking/bgp-exported versions = all
28592025-04-30T06:43:45.579ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/networking/bgp-message-history versions = all
28602025-04-30T06:43:45.579ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/networking/bgp-routes-ipv4 versions = all
28612025-04-30T06:43:45.579ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/networking/bgp-status versions = all
28622025-04-30T06:43:45.579ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/networking/loopback-address versions = all
28632025-04-30T06:43:45.579ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/system/networking/loopback-address versions = all
28642025-04-30T06:43:45.580ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = DELETE path = /v1/system/networking/loopback-address/{rack_id}/{switch_location}/{address}/{subnet_mask} versions = all
28652025-04-30T06:43:45.580ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = DELETE path = /v1/system/networking/switch-port-settings versions = all
28662025-04-30T06:43:45.580ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/networking/switch-port-settings versions = all
28672025-04-30T06:43:45.580ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/system/networking/switch-port-settings versions = all
28682025-04-30T06:43:45.580ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/networking/switch-port-settings/{port} versions = all
28692025-04-30T06:43:45.580ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/policy versions = all
28702025-04-30T06:43:45.580ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = PUT path = /v1/system/policy versions = all
28712025-04-30T06:43:45.580ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/roles versions = all
28722025-04-30T06:43:45.580ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/roles/{role_name} versions = all
28732025-04-30T06:43:45.580ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/silo-quotas versions = all
28742025-04-30T06:43:45.581ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/silos versions = all
28752025-04-30T06:43:45.581ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/system/silos versions = all
28762025-04-30T06:43:45.581ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = DELETE path = /v1/system/silos/{silo} versions = all
28772025-04-30T06:43:45.581ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/silos/{silo} versions = all
28782025-04-30T06:43:45.581ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/silos/{silo}/ip-pools versions = all
28792025-04-30T06:43:45.581ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/silos/{silo}/policy versions = all
28802025-04-30T06:43:45.581ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = PUT path = /v1/system/silos/{silo}/policy versions = all
28812025-04-30T06:43:45.581ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/silos/{silo}/quotas versions = all
28822025-04-30T06:43:45.581ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = PUT path = /v1/system/silos/{silo}/quotas versions = all
28832025-04-30T06:43:45.581ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/system/timeseries/query versions = all
28842025-04-30T06:43:45.581ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/timeseries/schemas versions = all
28852025-04-30T06:43:45.581ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = PUT path = /v1/system/update/repository versions = all
28862025-04-30T06:43:45.582ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/update/repository/{system_version} versions = all
28872025-04-30T06:43:45.582ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/update/target-release versions = all
28882025-04-30T06:43:45.582ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = PUT path = /v1/system/update/target-release versions = all
28892025-04-30T06:43:45.582ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/users versions = all
28902025-04-30T06:43:45.582ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/users/{user_id} versions = all
28912025-04-30T06:43:45.582ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/users-builtin versions = all
28922025-04-30T06:43:45.582ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/users-builtin/{user} versions = all
28932025-04-30T06:43:45.582ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/utilization/silos versions = all
28942025-04-30T06:43:45.583ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/system/utilization/silos/{silo} versions = all
28952025-04-30T06:43:45.583ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/timeseries/query versions = all
28962025-04-30T06:43:45.583ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/users versions = all
28972025-04-30T06:43:45.583ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/utilization versions = all
28982025-04-30T06:43:45.583ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/vpc-firewall-rules versions = all
28992025-04-30T06:43:45.584ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = PUT path = /v1/vpc-firewall-rules versions = all
29002025-04-30T06:43:45.584ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/vpc-router-routes versions = all
29012025-04-30T06:43:45.584ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/vpc-router-routes versions = all
29022025-04-30T06:43:45.584ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = DELETE path = /v1/vpc-router-routes/{route} versions = all
29032025-04-30T06:43:45.584ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/vpc-router-routes/{route} versions = all
29042025-04-30T06:43:45.584ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = PUT path = /v1/vpc-router-routes/{route} versions = all
29052025-04-30T06:43:45.584ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/vpc-routers versions = all
29062025-04-30T06:43:45.584ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/vpc-routers versions = all
29072025-04-30T06:43:45.584ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = DELETE path = /v1/vpc-routers/{router} versions = all
29082025-04-30T06:43:45.584ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/vpc-routers/{router} versions = all
29092025-04-30T06:43:45.584ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = PUT path = /v1/vpc-routers/{router} versions = all
29102025-04-30T06:43:45.584ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/vpc-subnets versions = all
29112025-04-30T06:43:45.585ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/vpc-subnets versions = all
29122025-04-30T06:43:45.585ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = DELETE path = /v1/vpc-subnets/{subnet} versions = all
29132025-04-30T06:43:45.585ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/vpc-subnets/{subnet} versions = all
29142025-04-30T06:43:45.585ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = PUT path = /v1/vpc-subnets/{subnet} versions = all
29152025-04-30T06:43:45.585ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/vpc-subnets/{subnet}/network-interfaces versions = all
29162025-04-30T06:43:45.585ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/vpcs versions = all
29172025-04-30T06:43:45.585ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/vpcs versions = all
29182025-04-30T06:43:45.586ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = DELETE path = /v1/vpcs/{vpc} versions = all
29192025-04-30T06:43:45.586ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/vpcs/{vpc} versions = all
29202025-04-30T06:43:45.586ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = PUT path = /v1/vpcs/{vpc} versions = all
29212025-04-30T06:43:45.586ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/webhooks/deliveries versions = all
29222025-04-30T06:43:45.586ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/webhooks/deliveries/{event_id}/resend versions = all
29232025-04-30T06:43:45.586ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/webhooks/event-classes versions = all
29242025-04-30T06:43:45.586ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/webhooks/receivers versions = all
29252025-04-30T06:43:45.586ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/webhooks/receivers versions = all
29262025-04-30T06:43:45.587ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = DELETE path = /v1/webhooks/receivers/{receiver} versions = all
29272025-04-30T06:43:45.587ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/webhooks/receivers/{receiver} versions = all
29282025-04-30T06:43:45.587ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = PUT path = /v1/webhooks/receivers/{receiver} versions = all
29292025-04-30T06:43:45.587ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/webhooks/receivers/{receiver}/probe versions = all
29302025-04-30T06:43:45.587ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/webhooks/receivers/{receiver}/subscriptions versions = all
29312025-04-30T06:43:45.587ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = DELETE path = /v1/webhooks/receivers/{receiver}/subscriptions/{subscription} versions = all
29322025-04-30T06:43:45.587ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = GET path = /v1/webhooks/secrets versions = all
29332025-04-30T06:43:45.588ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = POST path = /v1/webhooks/secrets versions = all
29342025-04-30T06:43:45.588ZDEBGtest_unauthorized (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:51935 method = DELETE path = /v1/webhooks/secrets/{secret_id} versions = all
29352025-04-30T06:43:45.588ZINFOtest_unauthorized (dropshot_external_techport): listening local_addr = 127.0.0.1:51935
29362025-04-30T06:43:45.589ZDEBGtest_unauthorized (dropshot_external_techport): successfully registered DTrace USDT probes local_addr = 127.0.0.1:51935
29372025-04-30T06:43:45.590ZDEBGtest_unauthorized (nexus-producer-server): registered DTrace probes
29382025-04-30T06:43:45.590ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = 127.0.0.1:53884 method = GET path = /{producer_id} versions = all
29392025-04-30T06:43:45.590ZINFOtest_unauthorized (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:328 local_addr = 127.0.0.1:53884
29402025-04-30T06:43:45.591ZDEBGtest_unauthorized (dropshot): successfully registered DTrace USDT probes local_addr = 127.0.0.1:53884
29412025-04-30T06:43:45.591ZDEBGtest_unauthorized (nexus-producer-server): Requested any available port, Dropshot server has been bound to 127.0.0.1:53884
29422025-04-30T06:43:45.591ZDEBGtest_unauthorized (nexus-producer-server): Nexus IP provided explicitly, will use it addr = 127.0.0.1:34029
29432025-04-30T06:43:45.591ZDEBGtest_unauthorized (nexus-producer-server): starting producer registration task
29442025-04-30T06:43:45.592ZINFOtest_unauthorized (nexus-producer-server): starting oximeter metric producer server address = 127.0.0.1:53884 file = oximeter/producer/src/lib.rs:279 interval = 10s producer_id = 8d752adb-08b9-405f-a11b-66019bf447b5
29452025-04-30T06:43:45.592ZDEBGtest_unauthorized (init_with_steps): Running step start_oximeter
29462025-04-30T06:43:45.592ZDEBGtest_unauthorized: Starting Oximeter
29472025-04-30T06:43:45.593ZDEBGtest_unauthorized (oximeter): registered DTrace probes
29482025-04-30T06:43:45.593ZINFOtest_unauthorized (oximeter): starting oximeter server file = oximeter/collector/src/lib.rs:236
29492025-04-30T06:43:45.593ZDEBGtest_unauthorized (oximeter): creating ClickHouse client
29502025-04-30T06:43:45.594ZDEBGtest_unauthorized (clickhouse-client): registered USDT probes collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a
29512025-04-30T06:43:45.595ZDEBGtest_unauthorized (producer-registration-task): registering as oximeter producer with Nexus registration_count = 0
29522025-04-30T06:43:45.595ZDEBGtest_unauthorized (producer-registration-task): will register with Nexus at 127.0.0.1:34029
29532025-04-30T06:43:45.657ZDEBGtest_unauthorized (producer-registration-task): client request body = Some(Body) method = POST uri = http://127.0.0.1:34029/metrics/producers
29542025-04-30T06:43:45.659ZINFOtest_unauthorized (dropshot_internal): accepted connection local_addr = 127.0.0.1:34029 remote_addr = 127.0.0.1:50376
29552025-04-30T06:43:45.663ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "43e16b06-fd28-40b2-903e-73d77f17efb3", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:45 GMT"} })
29562025-04-30T06:43:45.664ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
29572025-04-30T06:43:45.667ZTRACtest_unauthorized (dropshot_internal): incoming request local_addr = 127.0.0.1:34029 method = POST remote_addr = 127.0.0.1:50376 req_id = 2efa59d0-68d3-467d-b368-86ac33753902 uri = /metrics/producers
29582025-04-30T06:43:45.668ZTRACtest_unauthorized (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = POST remote_addr = 127.0.0.1:50376 req_id = 2efa59d0-68d3-467d-b368-86ac33753902 resource = Database uri = /metrics/producers
29592025-04-30T06:43:45.668ZDEBGtest_unauthorized (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = POST remote_addr = 127.0.0.1:50376 req_id = 2efa59d0-68d3-467d-b368-86ac33753902 roles = RoleSet { roles: {} } uri = /metrics/producers
29602025-04-30T06:43:45.669ZDEBGtest_unauthorized (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = POST remote_addr = 127.0.0.1:50376 req_id = 2efa59d0-68d3-467d-b368-86ac33753902 resource = Database result = Ok(()) uri = /metrics/producers
29612025-04-30T06:43:45.672ZTRACtest_unauthorized (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a sql = SELECT MAX(value) AS max_version FROM oximeter.version;
29622025-04-30T06:43:45.685ZWARNtest_unauthorized (clickhouse-client): oximeter database does not exist, or is out-of-date collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/db/src/client/mod.rs:894 id = 02357a36-53b1-4e9c-b024-19b15e97585a
29632025-04-30T06:43:45.685ZDEBGtest_unauthorized (oximeter-agent): oximeter database does not exist, creating collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
29642025-04-30T06:43:45.685ZINFOtest_unauthorized (clickhouse-client): reading db version collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/db/src/client/mod.rs:794 id = 02357a36-53b1-4e9c-b024-19b15e97585a
29652025-04-30T06:43:45.686ZTRACtest_unauthorized (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a sql = SELECT MAX(value) AS max_version FROM oximeter.version;
29662025-04-30T06:43:45.689ZWARNtest_unauthorized (clickhouse-client): oximeter database does not exist, or is out-of-date collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/db/src/client/mod.rs:894 id = 02357a36-53b1-4e9c-b024-19b15e97585a
29672025-04-30T06:43:45.689ZINFOtest_unauthorized (clickhouse-client): read oximeter database version collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/db/src/client/mod.rs:799 id = 02357a36-53b1-4e9c-b024-19b15e97585a version = 0
29682025-04-30T06:43:45.689ZINFOtest_unauthorized (clickhouse-client): initializing oximeter schema collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/db/src/client/mod.rs:804 id = 02357a36-53b1-4e9c-b024-19b15e97585a
29692025-04-30T06:43:45.689ZDEBGtest_unauthorized (clickhouse-client): initializing ClickHouse database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a
29702025-04-30T06:43:45.691ZTRACtest_unauthorized (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a sql = CREATE DATABASE IF NOT EXISTS oximeter
29712025-04-30T06:43:45.696ZTRACtest_unauthorized (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a sql = \n\n/* The version table contains metadata about the `oximeter` database */\nCREATE TABLE IF NOT EXISTS oximeter.version\n(\n value UInt64,\n timestamp DateTime64(9, 'UTC')\n)\nENGINE = MergeTree()\nORDER BY (value, timestamp)
29722025-04-30T06:43:45.699ZTRACtest_unauthorized (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a sql = \n\n/* The measurement tables contain all individual samples from each timeseries.\n *\n * Each table stores a single datum type, and otherwise contains nearly the same\n * structure. The primary sorting key is on the timeseries name, key, and then\n * timestamp, so that all timeseries from the same schema are grouped, followed\n * by all samples from the same timeseries.\n *\n * This reflects that one usually looks up the _key_ in one or more field table,\n * and then uses that to index quickly into the measurements tables.\n */\nCREATE TABLE IF NOT EXISTS oximeter.measurements_bool\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Bool)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29732025-04-30T06:43:45.705ZTRACtest_unauthorized (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_i8\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Int8)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29742025-04-30T06:43:45.712ZTRACtest_unauthorized (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_u8\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(UInt8)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29752025-04-30T06:43:45.715ZINFOtest_unauthorized (dropshot_internal): request completed error_message_external = Service Unavailable error_message_internal = no Oximeter instances available for assignment latency_us = 48608 local_addr = 127.0.0.1:34029 method = POST remote_addr = 127.0.0.1:50376 req_id = 2efa59d0-68d3-467d-b368-86ac33753902 response_code = 503 uri = /metrics/producers
29762025-04-30T06:43:45.717ZTRACtest_unauthorized (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_i16\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Int16)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29772025-04-30T06:43:45.718ZDEBGtest_unauthorized (producer-registration-task): client response result = Ok(Response { url: "http://127.0.0.1:34029/metrics/producers", status: 503, headers: {"content-type": "application/json", "x-request-id": "2efa59d0-68d3-467d-b368-86ac33753902", "content-length": "133", "date": "Wed, 30 Apr 2025 06:43:45 GMT"} })
29782025-04-30T06:43:45.720ZWARNtest_unauthorized (producer-registration-task): failed to register with Nexus, will retry count = 0 delay = 125.588587ms error = "Error Response: status: 503 Service Unavailable; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"2efa59d0-68d3-467d-b368-86ac33753902\\", \\"content-length\\": \\"133\\", \\"date\\": \\"Wed, 30 Apr 2025 06:43:45 GMT\\"}; value: Error { error_code: Some(\\"ServiceNotAvailable\\"), message: \\"Service Unavailable\\", request_id: \\"2efa59d0-68d3-467d-b368-86ac33753902\\" }" file = oximeter/producer/src/lib.rs:371
29792025-04-30T06:43:45.721ZTRACtest_unauthorized (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_u16\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(UInt16)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29802025-04-30T06:43:45.725ZTRACtest_unauthorized (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_i32\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Int32)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29812025-04-30T06:43:45.726ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cdf5005f-882b-4c1a-b684-df240268e551", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:45 GMT"} })
29822025-04-30T06:43:45.726ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
29832025-04-30T06:43:45.729ZTRACtest_unauthorized (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_u32\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(UInt32)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29842025-04-30T06:43:45.746ZTRACtest_unauthorized (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_i64\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Int64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29852025-04-30T06:43:45.749ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b81ed9c9-fbf2-48ae-8c37-992887c7d1e7", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:45 GMT"} })
29862025-04-30T06:43:45.749ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
29872025-04-30T06:43:45.750ZTRACtest_unauthorized (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_u64\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(UInt64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29882025-04-30T06:43:45.752ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "72ea4ac9-6098-43d0-b0a7-efc8f4910bc4", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:45 GMT"} })
29892025-04-30T06:43:45.752ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
29902025-04-30T06:43:45.754ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f13480e9-ad8c-44b5-bdfe-39a4f77426af", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:45 GMT"} })
29912025-04-30T06:43:45.754ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
29922025-04-30T06:43:45.754ZTRACtest_unauthorized (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_f32\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Float32)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29932025-04-30T06:43:45.755ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ee63af38-7411-4e15-a02e-4afb6330f6e3", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:45 GMT"} })
29942025-04-30T06:43:45.755ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
29952025-04-30T06:43:45.757ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a3ab4fae-93ee-4569-903f-8bfe031a44af", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:45 GMT"} })
29962025-04-30T06:43:45.757ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
29972025-04-30T06:43:45.759ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e9cde8aa-1dfb-422e-9610-9c70cf463329", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:45 GMT"} })
29982025-04-30T06:43:45.759ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
29992025-04-30T06:43:45.759ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
30002025-04-30T06:43:45.760ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
30012025-04-30T06:43:45.760ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
30022025-04-30T06:43:45.762ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
30032025-04-30T06:43:45.763ZTRACtest_unauthorized (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_f64\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30042025-04-30T06:43:45.765ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30052025-04-30T06:43:45.782ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
30062025-04-30T06:43:45.782ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
30072025-04-30T06:43:45.782ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
30082025-04-30T06:43:45.782ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
30092025-04-30T06:43:45.782ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
30102025-04-30T06:43:45.790ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
30112025-04-30T06:43:45.791ZTRACtest_unauthorized (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_string\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(String)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30122025-04-30T06:43:45.797ZTRACtest_unauthorized (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_bytes\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n /*\n * NOTE: Right now we can't unambiguously record a nullable byte array.\n * Arrays cannot be nested in `Nullable()` types, and encoding the array as\n * a string isn't palatable for a few reasons.\n * See: https://github.com/oxidecomputer/omicron/issues/4551 for more\n * details.\n */\n datum Array(UInt8)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30132025-04-30T06:43:45.802ZTRACtest_unauthorized (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativei64\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Int64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30142025-04-30T06:43:45.803ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30152025-04-30T06:43:45.808ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
30162025-04-30T06:43:45.808ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
30172025-04-30T06:43:45.808ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
30182025-04-30T06:43:45.808ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
30192025-04-30T06:43:45.813ZTRACtest_unauthorized (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativeu64\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(UInt64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30202025-04-30T06:43:45.817ZTRACtest_unauthorized (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativef32\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Float32)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30212025-04-30T06:43:45.909ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
30222025-04-30T06:43:45.909ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory
30232025-04-30T06:43:45.910ZTRACtest_unauthorized (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a sql = \n\n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativef64\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30242025-04-30T06:43:45.913ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:33619 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
30252025-04-30T06:43:45.915ZTRACtest_unauthorized (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogrami8\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n /*\n * NOTE: Array types cannot be Nullable, see\n * https://clickhouse.com/docs/en/sql-reference/data-types/nullable\n * for more details.\n *\n * This means we need to use empty arrays to indicate a missing value. This\n * is unfortunate, and at this point relies on the fact that an\n * `oximeter::Histogram` cannot have zero bins. If that changes, we'll need\n * to figure out another way to represent missing samples here.\n */\n bins Array(Int8),\n counts Array(UInt64),\n min Int8,\n max Int8,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30262025-04-30T06:43:45.918ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:33619 req_id = 8f9c45e8-d39b-409f-a8bd-4c1013b17920 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
30272025-04-30T06:43:45.919ZDEBGtest_unauthorized (producer-registration-task): will register with Nexus at 127.0.0.1:34029
30282025-04-30T06:43:45.989ZDEBGtest_unauthorized (producer-registration-task): client request body = Some(Body) method = POST uri = http://127.0.0.1:34029/metrics/producers
30292025-04-30T06:43:45.990ZINFOtest_unauthorized (dropshot): request completed latency_us = 72143 local_addr = [::1]:57786 method = GET remote_addr = [::1]:33619 req_id = 8f9c45e8-d39b-409f-a8bd-4c1013b17920 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
30302025-04-30T06:43:45.990ZTRACtest_unauthorized (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogramu8\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(UInt8),\n counts Array(UInt64),\n min UInt8,\n max UInt8,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30312025-04-30T06:43:45.990ZINFOtest_unauthorized (dropshot_internal): accepted connection local_addr = 127.0.0.1:34029 remote_addr = 127.0.0.1:37706
30322025-04-30T06:43:45.991ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = http://[::1]:57786 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:57786/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "8f9c45e8-d39b-409f-a8bd-4c1013b17920", "content-length": "2769", "date": "Wed, 30 Apr 2025 06:43:45 GMT"} })
30332025-04-30T06:43:45.993ZDEBGtest_unauthorized (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
30342025-04-30T06:43:45.993ZDEBGtest_unauthorized (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
30352025-04-30T06:43:45.993ZDEBGtest_unauthorized (BackgroundTasks): finished collection background_task = inventory_collection
30362025-04-30T06:43:45.995ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
30372025-04-30T06:43:45.995ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
30382025-04-30T06:43:45.995ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
30392025-04-30T06:43:45.995ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
30402025-04-30T06:43:45.996ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
30412025-04-30T06:43:45.998ZTRACtest_unauthorized (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogrami16\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(Int16),\n counts Array(UInt64),\n min Int16,\n max Int16,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30422025-04-30T06:43:45.998ZTRACtest_unauthorized (dropshot_internal): incoming request local_addr = 127.0.0.1:34029 method = POST remote_addr = 127.0.0.1:37706 req_id = 5d22fc70-797f-4583-8195-1e463f07fa2e uri = /metrics/producers
30432025-04-30T06:43:45.999ZTRACtest_unauthorized (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = POST remote_addr = 127.0.0.1:37706 req_id = 5d22fc70-797f-4583-8195-1e463f07fa2e resource = Database uri = /metrics/producers
30442025-04-30T06:43:45.999ZDEBGtest_unauthorized (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = POST remote_addr = 127.0.0.1:37706 req_id = 5d22fc70-797f-4583-8195-1e463f07fa2e roles = RoleSet { roles: {} } uri = /metrics/producers
30452025-04-30T06:43:45.999ZDEBGtest_unauthorized (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = POST remote_addr = 127.0.0.1:37706 req_id = 5d22fc70-797f-4583-8195-1e463f07fa2e resource = Database result = Ok(()) uri = /metrics/producers
30462025-04-30T06:43:46.002ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30472025-04-30T06:43:46.007ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
30482025-04-30T06:43:46.008ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
30492025-04-30T06:43:46.008ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
30502025-04-30T06:43:46.008ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
30512025-04-30T06:43:46.015ZTRACtest_unauthorized (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogramu16\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(UInt16),\n counts Array(UInt64),\n min UInt16,\n max UInt16,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30522025-04-30T06:43:46.016ZTRACtest_unauthorized (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogrami32\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(Int32),\n counts Array(UInt64),\n min Int32,\n max Int32,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30532025-04-30T06:43:46.050ZTRACtest_unauthorized (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogramu32\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(UInt32),\n counts Array(UInt64),\n min UInt32,\n max UInt32,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30542025-04-30T06:43:46.061ZINFOtest_unauthorized (dropshot_internal): request completed error_message_external = Service Unavailable error_message_internal = no Oximeter instances available for assignment latency_us = 62496 local_addr = 127.0.0.1:34029 method = POST remote_addr = 127.0.0.1:37706 req_id = 5d22fc70-797f-4583-8195-1e463f07fa2e response_code = 503 uri = /metrics/producers
30552025-04-30T06:43:46.061ZDEBGtest_unauthorized (producer-registration-task): client response result = Ok(Response { url: "http://127.0.0.1:34029/metrics/producers", status: 503, headers: {"content-type": "application/json", "x-request-id": "5d22fc70-797f-4583-8195-1e463f07fa2e", "content-length": "133", "date": "Wed, 30 Apr 2025 06:43:46 GMT"} })
30562025-04-30T06:43:46.064ZWARNtest_unauthorized (producer-registration-task): failed to register with Nexus, will retry count = 1 delay = 468.698912ms error = "Error Response: status: 503 Service Unavailable; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"5d22fc70-797f-4583-8195-1e463f07fa2e\\", \\"content-length\\": \\"133\\", \\"date\\": \\"Wed, 30 Apr 2025 06:43:46 GMT\\"}; value: Error { error_code: Some(\\"ServiceNotAvailable\\"), message: \\"Service Unavailable\\", request_id: \\"5d22fc70-797f-4583-8195-1e463f07fa2e\\" }" file = oximeter/producer/src/lib.rs:371
30572025-04-30T06:43:46.065ZTRACtest_unauthorized (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogrami64\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(Int64),\n counts Array(UInt64),\n min Int64,\n max Int64,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30582025-04-30T06:43:46.075ZTRACtest_unauthorized (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogramu64\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(UInt64),\n counts Array(UInt64),\n min UInt64,\n max UInt64,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30592025-04-30T06:43:46.078ZTRACtest_unauthorized (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogramf32\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(Float32),\n counts Array(UInt64),\n min Float32,\n max Float32,\n sum_of_samples Float64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30602025-04-30T06:43:46.085ZTRACtest_unauthorized (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogramf64\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(Float64),\n counts Array(UInt64),\n min Float64,\n max Float64,\n sum_of_samples Float64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30612025-04-30T06:43:46.089ZTRACtest_unauthorized (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a sql = \n\n/* The field tables store named dimensions of each timeseries.\n *\n * As with the measurement tables, there is one field table for each field data\n * type. Fields are deduplicated by using the "replacing merge tree", though\n * this behavior **must not** be relied upon for query correctness.\n *\n * The index for the fields differs from the measurements, however. Rows are\n * sorted by timeseries name, then field name, field value, and finally\n * timeseries key. This reflects the most common pattern for looking them up:\n * by field name and possibly value, within a timeseries. The resulting keys are\n * usually then used to look up measurements.\n *\n * NOTE: We may want to consider a secondary index on these tables, sorting by\n * timeseries name and then key, since it would improve lookups where one\n * already has the key. Realistically though, these tables are quite small and\n * so performance benefits will be low in absolute terms.\n *\n * TTL: We use a materialized column to expire old field table records. This\n * column is generated automatically by the database whenever a new row is\n * inserted. It cannot be inserted directly, nor is it returned in a `SELECT *`\n * query. Since these tables are `ReplacingMergeTree`s, that means the last\n * record will remain during a deduplication, which will have the last\n * timestamp. ClickHouse will then expire old data for us, similar to the\n * measurement tables.\n */\nCREATE TABLE IF NOT EXISTS oximeter.fields_bool\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value Bool,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30622025-04-30T06:43:46.097ZTRACtest_unauthorized (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_i8\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value Int8,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30632025-04-30T06:43:46.108ZTRACtest_unauthorized (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_u8\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value UInt8,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30642025-04-30T06:43:46.122ZTRACtest_unauthorized (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_i16\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value Int16,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30652025-04-30T06:43:46.125ZTRACtest_unauthorized (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_u16\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value UInt16,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30662025-04-30T06:43:46.128ZTRACtest_unauthorized (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_i32\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value Int32,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30672025-04-30T06:43:46.131ZTRACtest_unauthorized (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_u32\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value UInt32,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30682025-04-30T06:43:46.133ZTRACtest_unauthorized (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_i64\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value Int64,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30692025-04-30T06:43:46.136ZTRACtest_unauthorized (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_u64\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value UInt64,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30702025-04-30T06:43:46.141ZTRACtest_unauthorized (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_ipaddr\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value IPv6,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30712025-04-30T06:43:46.146ZTRACtest_unauthorized (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_string\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value String,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30722025-04-30T06:43:46.150ZTRACtest_unauthorized (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_uuid\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value UUID,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30732025-04-30T06:43:46.154ZTRACtest_unauthorized (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a sql = \n\n/* The timeseries schema table stores the extracted schema for the samples\n * oximeter collects.\n */\nCREATE TABLE IF NOT EXISTS oximeter.timeseries_schema\n(\n timeseries_name String,\n fields Nested(\n name String,\n type Enum(\n 'Bool' = 1,\n 'I64' = 2,\n 'IpAddr' = 3,\n 'String' = 4,\n 'Uuid' = 6,\n 'I8' = 7,\n 'U8' = 8,\n 'I16' = 9,\n 'U16' = 10,\n 'I32' = 11,\n 'U32' = 12,\n 'U64' = 13\n ),\n source Enum(\n 'Target' = 1,\n 'Metric' = 2\n )\n ),\n datum_type Enum(\n 'Bool' = 1,\n 'I64' = 2,\n 'F64' = 3,\n 'String' = 4,\n 'Bytes' = 5,\n 'CumulativeI64' = 6,\n 'CumulativeF64' = 7,\n 'HistogramI64' = 8,\n 'HistogramF64' = 9,\n 'I8' = 10,\n 'U8' = 11,\n 'I16' = 12,\n 'U16' = 13,\n 'I32' = 14,\n 'U32' = 15,\n 'U64' = 16,\n 'F32' = 17,\n 'CumulativeU64' = 18,\n 'CumulativeF32' = 19,\n 'HistogramI8' = 20,\n 'HistogramU8' = 21,\n 'HistogramI16' = 22,\n 'HistogramU16' = 23,\n 'HistogramI32' = 24,\n 'HistogramU32' = 25,\n 'HistogramU64' = 26,\n 'HistogramF32' = 27\n ),\n created DateTime64(9, 'UTC')\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, fields.name)
30742025-04-30T06:43:46.154ZWARNtest_unauthorized (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
30752025-04-30T06:43:46.155ZWARNtest_unauthorized (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
30762025-04-30T06:43:46.155ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 2.153805678s iteration = 3 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
30772025-04-30T06:43:46.163ZINFOtest_unauthorized (clickhouse-client): inserting current version collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/db/src/client/mod.rs:825 id = 02357a36-53b1-4e9c-b024-19b15e97585a version = 14
30782025-04-30T06:43:46.163ZTRACtest_unauthorized (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a sql = INSERT INTO oximeter.version (*) VALUES (14, now());
30792025-04-30T06:43:46.169ZDEBGtest_unauthorized (clickhouse-client): registered USDT probes collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5b02530d-2c01-4e55-9f2b-c2ef0d6de440
30802025-04-30T06:43:46.170ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:65069 method = GET path = /info versions = all
30812025-04-30T06:43:46.170ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:65069 method = GET path = /producers versions = all
30822025-04-30T06:43:46.170ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:65069 method = DELETE path = /producers/{producer_id} versions = all
30832025-04-30T06:43:46.170ZDEBGtest_unauthorized (dropshot): registered endpoint local_addr = [::1]:65069 method = GET path = /producers/{producer_id} versions = all
30842025-04-30T06:43:46.170ZINFOtest_unauthorized (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:328 local_addr = [::1]:65069
30852025-04-30T06:43:46.171ZDEBGtest_unauthorized (dropshot): successfully registered DTrace USDT probes local_addr = [::1]:65069
30862025-04-30T06:43:46.172ZDEBGtest_unauthorized (oximeter): registered USDT probes
30872025-04-30T06:43:46.172ZDEBGtest_unauthorized (oximeter): contacting nexus
30882025-04-30T06:43:46.936ZDEBGtest_unauthorized (oximeter): client request body = Some(Body) method = POST uri = http://127.0.0.1:34029/metrics/collectors
30892025-04-30T06:43:46.939ZDEBGtest_unauthorized (producer-registration-task): will register with Nexus at 127.0.0.1:34029
30902025-04-30T06:43:47.022ZDEBGtest_unauthorized (producer-registration-task): client request body = Some(Body) method = POST uri = http://127.0.0.1:34029/metrics/producers
30912025-04-30T06:43:47.025ZINFOtest_unauthorized (dropshot_internal): accepted connection local_addr = 127.0.0.1:34029 remote_addr = 127.0.0.1:40698
30922025-04-30T06:43:47.025ZINFOtest_unauthorized (dropshot_internal): accepted connection local_addr = 127.0.0.1:34029 remote_addr = 127.0.0.1:52364
30932025-04-30T06:43:47.034ZTRACtest_unauthorized (dropshot_internal): incoming request local_addr = 127.0.0.1:34029 method = POST remote_addr = 127.0.0.1:52364 req_id = f5695c72-8af3-4f59-a0fb-9c8cb086d6ed uri = /metrics/producers
30942025-04-30T06:43:47.034ZTRACtest_unauthorized (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = POST remote_addr = 127.0.0.1:52364 req_id = f5695c72-8af3-4f59-a0fb-9c8cb086d6ed resource = Database uri = /metrics/producers
30952025-04-30T06:43:47.034ZDEBGtest_unauthorized (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = POST remote_addr = 127.0.0.1:52364 req_id = f5695c72-8af3-4f59-a0fb-9c8cb086d6ed roles = RoleSet { roles: {} } uri = /metrics/producers
30962025-04-30T06:43:47.035ZDEBGtest_unauthorized (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = POST remote_addr = 127.0.0.1:52364 req_id = f5695c72-8af3-4f59-a0fb-9c8cb086d6ed resource = Database result = Ok(()) uri = /metrics/producers
30972025-04-30T06:43:47.037ZTRACtest_unauthorized (dropshot_internal): incoming request local_addr = 127.0.0.1:34029 method = POST remote_addr = 127.0.0.1:40698 req_id = 5cbcbd6e-33e3-4378-bfb7-25e7426e3177 uri = /metrics/collectors
30982025-04-30T06:43:47.038ZTRACtest_unauthorized (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = POST remote_addr = 127.0.0.1:40698 req_id = 5cbcbd6e-33e3-4378-bfb7-25e7426e3177 resource = Database uri = /metrics/collectors
30992025-04-30T06:43:47.038ZDEBGtest_unauthorized (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = POST remote_addr = 127.0.0.1:40698 req_id = 5cbcbd6e-33e3-4378-bfb7-25e7426e3177 roles = RoleSet { roles: {} } uri = /metrics/collectors
31002025-04-30T06:43:47.040ZDEBGtest_unauthorized (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = POST remote_addr = 127.0.0.1:40698 req_id = 5cbcbd6e-33e3-4378-bfb7-25e7426e3177 resource = Database result = Ok(()) uri = /metrics/collectors
31012025-04-30T06:43:47.064ZINFOtest_unauthorized (dropshot_internal): request completed error_message_external = Service Unavailable error_message_internal = no Oximeter instances available for assignment latency_us = 30724 local_addr = 127.0.0.1:34029 method = POST remote_addr = 127.0.0.1:52364 req_id = f5695c72-8af3-4f59-a0fb-9c8cb086d6ed response_code = 503 uri = /metrics/producers
31022025-04-30T06:43:47.074ZINFOtest_unauthorized (nexus): registered new oximeter metric collection server address = [::1]:65069 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78
31032025-04-30T06:43:47.079ZINFOtest_unauthorized (dropshot_internal): request completed latency_us = 42026 local_addr = 127.0.0.1:34029 method = POST remote_addr = 127.0.0.1:40698 req_id = 5cbcbd6e-33e3-4378-bfb7-25e7426e3177 response_code = 204 uri = /metrics/collectors
31042025-04-30T06:43:47.084ZDEBGtest_unauthorized (producer-registration-task): client response result = Ok(Response { url: "http://127.0.0.1:34029/metrics/producers", status: 503, headers: {"content-type": "application/json", "x-request-id": "f5695c72-8af3-4f59-a0fb-9c8cb086d6ed", "content-length": "133", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
31052025-04-30T06:43:47.088ZWARNtest_unauthorized (producer-registration-task): failed to register with Nexus, will retry count = 2 delay = 1.492722973s error = "Error Response: status: 503 Service Unavailable; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"f5695c72-8af3-4f59-a0fb-9c8cb086d6ed\\", \\"content-length\\": \\"133\\", \\"date\\": \\"Wed, 30 Apr 2025 06:43:47 GMT\\"}; value: Error { error_code: Some(\\"ServiceNotAvailable\\"), message: \\"Service Unavailable\\", request_id: \\"f5695c72-8af3-4f59-a0fb-9c8cb086d6ed\\" }" file = oximeter/producer/src/lib.rs:371
31062025-04-30T06:43:47.089ZDEBGtest_unauthorized (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:34029/metrics/collectors", status: 204, headers: {"x-request-id": "5cbcbd6e-33e3-4378-bfb7-25e7426e3177", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
31072025-04-30T06:43:47.089ZINFOtest_unauthorized (oximeter): oximeter registered with nexus file = oximeter/collector/src/lib.rs:392 id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78
31082025-04-30T06:43:47.089ZDEBGtest_unauthorized (init_with_steps): Running step start_producer_server
31092025-04-30T06:43:47.089ZDEBGtest_unauthorized: Starting test metric Producer Server
31102025-04-30T06:43:47.095ZDEBGtest_unauthorized (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37512 method = GET path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
31112025-04-30T06:43:47.096ZDEBGtest_unauthorized (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37512 method = POST path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
31122025-04-30T06:43:47.096ZDEBGtest_unauthorized (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37512 method = DELETE path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
31132025-04-30T06:43:47.096ZDEBGtest_unauthorized (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37512 method = GET path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
31142025-04-30T06:43:47.096ZDEBGtest_unauthorized (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37512 method = GET path = /crucible/0/regions/{id}/snapshots server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
31152025-04-30T06:43:47.096ZDEBGtest_unauthorized (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37512 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
31162025-04-30T06:43:47.096ZDEBGtest_unauthorized (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37512 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
31172025-04-30T06:43:47.096ZDEBGtest_unauthorized (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37512 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
31182025-04-30T06:43:47.096ZDEBGtest_unauthorized (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37512 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
31192025-04-30T06:43:47.096ZINFOtest_unauthorized (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:37512 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
31202025-04-30T06:43:47.096ZDEBGtest_unauthorized (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:37512 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
31212025-04-30T06:43:47.097ZINFOtest_unauthorized (SledAgent): Created Simulated Crucible Server address = [::1]:37512 kind = storage server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
31222025-04-30T06:43:47.097ZINFOtest_unauthorized (nexus): inserting test physical disk model = test-model physical_disk_id = 81d10046-4d52-4950-96c0-d6403dc0d8a8 serial = totally-unique-serial: 81d10046-4d52-4950-96c0-d6403dc0d8a8 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 vendor = test-vendor
31232025-04-30T06:43:47.097ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
31242025-04-30T06:43:47.097ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
31252025-04-30T06:43:47.097ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
31262025-04-30T06:43:47.098ZINFOtest_unauthorized (oximeter-agent): starting refresh list task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:559
31272025-04-30T06:43:47.250ZINFOtest_unauthorized (dropshot_internal): accepted connection local_addr = 127.0.0.1:34029 remote_addr = 127.0.0.1:35816
31282025-04-30T06:43:47.250ZINFOtest_unauthorized (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
31292025-04-30T06:43:47.250ZTRACtest_unauthorized (results-sink): batch interval expired, but no samples to insert collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
31302025-04-30T06:43:47.250ZTRACtest_unauthorized (results-sink-cluster): batch interval expired, but no samples to insert collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
31312025-04-30T06:43:47.252ZDEBGtest_unauthorized (oximeter): client request body = None method = GET uri = http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
31322025-04-30T06:43:47.253ZTRACtest_unauthorized (dropshot_internal): incoming request local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = b6c9b128-bb7a-4964-83f3-614dcf468f15 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
31332025-04-30T06:43:47.253ZTRACtest_unauthorized (dropshot_internal): incoming request local_addr = 127.0.0.1:34029 method = POST remote_addr = 127.0.0.1:35816 req_id = 7f5a6f1d-a84b-428c-a0a2-d70bf794e2df uri = /metrics/producers
31342025-04-30T06:43:47.256ZTRACtest_unauthorized (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = b6c9b128-bb7a-4964-83f3-614dcf468f15 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
31352025-04-30T06:43:47.256ZDEBGtest_unauthorized (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = b6c9b128-bb7a-4964-83f3-614dcf468f15 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
31362025-04-30T06:43:47.256ZDEBGtest_unauthorized (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = b6c9b128-bb7a-4964-83f3-614dcf468f15 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
31372025-04-30T06:43:47.257ZTRACtest_unauthorized (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = POST remote_addr = 127.0.0.1:35816 req_id = 7f5a6f1d-a84b-428c-a0a2-d70bf794e2df resource = Database uri = /metrics/producers
31382025-04-30T06:43:47.257ZDEBGtest_unauthorized (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = POST remote_addr = 127.0.0.1:35816 req_id = 7f5a6f1d-a84b-428c-a0a2-d70bf794e2df roles = RoleSet { roles: {} } uri = /metrics/producers
31392025-04-30T06:43:47.257ZDEBGtest_unauthorized (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = POST remote_addr = 127.0.0.1:35816 req_id = 7f5a6f1d-a84b-428c-a0a2-d70bf794e2df resource = Database result = Ok(()) uri = /metrics/producers
31402025-04-30T06:43:47.258ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
31412025-04-30T06:43:47.268ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
31422025-04-30T06:43:47.269ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
31432025-04-30T06:43:47.273ZINFOtest_unauthorized (dropshot_internal): request completed latency_us = 20215 local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = b6c9b128-bb7a-4964-83f3-614dcf468f15 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
31442025-04-30T06:43:47.274ZDEBGtest_unauthorized (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "b6c9b128-bb7a-4964-83f3-614dcf468f15", "content-length": "29", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
31452025-04-30T06:43:47.274ZINFOtest_unauthorized (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 0 n_pruned_tasks = 0
31462025-04-30T06:43:47.280ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
31472025-04-30T06:43:47.285ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
31482025-04-30T06:43:47.285ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
31492025-04-30T06:43:47.285ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
31502025-04-30T06:43:47.285ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
31512025-04-30T06:43:47.297ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31522025-04-30T06:43:47.313ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
31532025-04-30T06:43:47.344ZINFOtest_unauthorized (nexus): assigned collector to new producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
31542025-04-30T06:43:47.344ZINFOtest_unauthorized (dropshot_internal): request completed latency_us = 91100 local_addr = 127.0.0.1:34029 method = POST remote_addr = 127.0.0.1:35816 req_id = 7f5a6f1d-a84b-428c-a0a2-d70bf794e2df response_code = 201 uri = /metrics/producers
31552025-04-30T06:43:47.471ZINFOtest_unauthorized (nexus): upserting zpool physical_disk_id = 81d10046-4d52-4950-96c0-d6403dc0d8a8 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 zpool_id = 4484a50e-e358-4130-8a2e-f161a1c4774f
31562025-04-30T06:43:47.471ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
31572025-04-30T06:43:47.471ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
31582025-04-30T06:43:47.471ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
31592025-04-30T06:43:47.473ZTRACtest_unauthorized (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
31602025-04-30T06:43:47.473ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
31612025-04-30T06:43:47.473ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
31622025-04-30T06:43:47.473ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
31632025-04-30T06:43:47.473ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
31642025-04-30T06:43:47.477ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31652025-04-30T06:43:47.479ZDEBGtest_unauthorized (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
31662025-04-30T06:43:47.491ZINFOtest_unauthorized (nexus): upserting Crucible dataset dataset_id = 74103528-1b19-4191-a2bf-86e6bc2bdd65 zpool_id = 4484a50e-e358-4130-8a2e-f161a1c4774f
31672025-04-30T06:43:47.499ZINFOtest_unauthorized (nexus): inserting test physical disk model = test-model physical_disk_id = 81d10046-4d52-4950-96c0-d6403dc0d8a8 serial = totally-unique-serial: 81d10046-4d52-4950-96c0-d6403dc0d8a8 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 vendor = test-vendor
31682025-04-30T06:43:47.500ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
31692025-04-30T06:43:47.500ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
31702025-04-30T06:43:47.500ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
31712025-04-30T06:43:47.504ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = PhysicalDisk { parent: Fleet, key: 81d10046-4d52-4950-96c0-d6403dc0d8a8 (physical_disk), lookup_type: ById(81d10046-4d52-4950-96c0-d6403dc0d8a8) }
31722025-04-30T06:43:47.504ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
31732025-04-30T06:43:47.504ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
31742025-04-30T06:43:47.504ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
31752025-04-30T06:43:47.505ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
31762025-04-30T06:43:47.507ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31772025-04-30T06:43:47.512ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = PhysicalDisk { parent: Fleet, key: 81d10046-4d52-4950-96c0-d6403dc0d8a8 (physical_disk), lookup_type: ById(81d10046-4d52-4950-96c0-d6403dc0d8a8) } result = Ok(())
31782025-04-30T06:43:47.512ZINFOtest_unauthorized (nexus): upserting zpool physical_disk_id = 81d10046-4d52-4950-96c0-d6403dc0d8a8 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 zpool_id = 4484a50e-e358-4130-8a2e-f161a1c4774f
31792025-04-30T06:43:47.512ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
31802025-04-30T06:43:47.512ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
31812025-04-30T06:43:47.512ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
31822025-04-30T06:43:47.515ZTRACtest_unauthorized (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
31832025-04-30T06:43:47.515ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
31842025-04-30T06:43:47.515ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
31852025-04-30T06:43:47.515ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
31862025-04-30T06:43:47.516ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
31872025-04-30T06:43:47.522ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31882025-04-30T06:43:47.524ZDEBGtest_unauthorized (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
31892025-04-30T06:43:47.530ZTRACtest_unauthorized (InternalApi): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
31902025-04-30T06:43:47.530ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
31912025-04-30T06:43:47.530ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
31922025-04-30T06:43:47.530ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
31932025-04-30T06:43:47.531ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
31942025-04-30T06:43:47.533ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31952025-04-30T06:43:47.536ZDEBGtest_unauthorized (InternalApi): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
31962025-04-30T06:43:47.536ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
31972025-04-30T06:43:47.536ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
31982025-04-30T06:43:47.537ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
31992025-04-30T06:43:47.545ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
32002025-04-30T06:43:47.545ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
32012025-04-30T06:43:47.545ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
32022025-04-30T06:43:47.545ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
32032025-04-30T06:43:47.545ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
32042025-04-30T06:43:47.546ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 4 reason = Signaled
32052025-04-30T06:43:47.546ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
32062025-04-30T06:43:47.546ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
32072025-04-30T06:43:47.546ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
32082025-04-30T06:43:47.546ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
32092025-04-30T06:43:47.546ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
32102025-04-30T06:43:47.554ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32112025-04-30T06:43:47.562ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
32122025-04-30T06:43:47.562ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
32132025-04-30T06:43:47.562ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
32142025-04-30T06:43:47.563ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
32152025-04-30T06:43:47.563ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32162025-04-30T06:43:47.571ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
32172025-04-30T06:43:47.571ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
32182025-04-30T06:43:47.571ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
32192025-04-30T06:43:47.571ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
32202025-04-30T06:43:47.581ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
32212025-04-30T06:43:47.581ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
32222025-04-30T06:43:47.581ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
32232025-04-30T06:43:47.592ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1a8b2318-abf8-406d-a389-112ab91a0de8, 0)]
32242025-04-30T06:43:47.592ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
32252025-04-30T06:43:47.609ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 5c c1 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 \\............_mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:64064 req_id = b0f3495e-e5e6-461d-a6ed-175f9091ae73
32262025-04-30T06:43:47.609ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 23745,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64064 req_id = b0f3495e-e5e6-461d-a6ed-175f9091ae73
32272025-04-30T06:43:47.609ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
32282025-04-30T06:43:47.609ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
32292025-04-30T06:43:47.609ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:64064 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = b0f3495e-e5e6-461d-a6ed-175f9091ae73
32302025-04-30T06:43:47.611ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4292, tv_nsec: 410070196 } })
32312025-04-30T06:43:47.611ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
32322025-04-30T06:43:47.614ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 76 3b 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e v;..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:63259 req_id = 602ce447-652c-4a4f-aa82-4f74d0e4b58d
32332025-04-30T06:43:47.615ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 30267,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63259 req_id = 602ce447-652c-4a4f-aa82-4f74d0e4b58d
32342025-04-30T06:43:47.615ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
32352025-04-30T06:43:47.615ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:63259 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 602ce447-652c-4a4f-aa82-4f74d0e4b58d
32362025-04-30T06:43:47.681ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
32372025-04-30T06:43:47.686ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: aa 73 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c .s..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:51341 req_id = 4d67f2c1-faad-4320-892e-b5698b1c3e37
32382025-04-30T06:43:47.686ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 43635,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51341 req_id = 4d67f2c1-faad-4320-892e-b5698b1c3e37
32392025-04-30T06:43:47.687ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
32402025-04-30T06:43:47.688ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 43635, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:51341 req_id = 4d67f2c1-faad-4320-892e-b5698b1c3e37
32412025-04-30T06:43:47.690ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
32422025-04-30T06:43:47.690ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
32432025-04-30T06:43:47.690ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
32442025-04-30T06:43:47.698ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "428fe6c4-32e4-4105-b89b-f7aa1072fc56", "content-length": "835", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
32452025-04-30T06:43:47.699ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
32462025-04-30T06:43:47.702ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "863b305a-6027-472a-8e07-8edf58257084", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
32472025-04-30T06:43:47.702ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
32482025-04-30T06:43:47.707ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a923d3f7-8bd1-4462-bc1d-f64d0915e8f3", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
32492025-04-30T06:43:47.707ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
32502025-04-30T06:43:47.709ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1fa1446c-85cd-458b-9d6c-3ea85e4fe54b", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
32512025-04-30T06:43:47.709ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
32522025-04-30T06:43:47.713ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cdbb5f0e-f25b-41d9-a884-a61b6390a9d1", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
32532025-04-30T06:43:47.713ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
32542025-04-30T06:43:47.716ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "17fa239a-d17b-4986-88e5-bfcc31246bab", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
32552025-04-30T06:43:47.717ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
32562025-04-30T06:43:47.719ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6c672ec8-d13a-4a38-b828-ce029f5086c5", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
32572025-04-30T06:43:47.720ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
32582025-04-30T06:43:47.726ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "124db055-83b9-4b57-8b2c-aebe5a59aad7", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
32592025-04-30T06:43:47.726ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
32602025-04-30T06:43:47.728ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0cc4484d-fc64-4c3f-be82-57e4585be386", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
32612025-04-30T06:43:47.728ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
32622025-04-30T06:43:47.729ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "dcd1d0d8-29f0-4849-8118-580bfedae509", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
32632025-04-30T06:43:47.729ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
32642025-04-30T06:43:47.730ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9923a361-e095-4775-b631-cff8efe51684", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
32652025-04-30T06:43:47.730ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
32662025-04-30T06:43:47.732ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7a659f41-6a25-4600-820a-c5a3e74dc14c", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
32672025-04-30T06:43:47.732ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
32682025-04-30T06:43:47.737ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4cebe5b1-b05e-43d9-88c9-046c11477716", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
32692025-04-30T06:43:47.737ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
32702025-04-30T06:43:47.743ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
32712025-04-30T06:43:47.747ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "deae640e-9705-493b-b37b-7fca54166d06", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
32722025-04-30T06:43:47.747ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
32732025-04-30T06:43:47.751ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bf77ea97-241a-4926-a185-c9102f7aa37c", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
32742025-04-30T06:43:47.751ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
32752025-04-30T06:43:47.755ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d573df94-05fb-4478-a5e8-6197eaec8b49", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
32762025-04-30T06:43:47.755ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
32772025-04-30T06:43:47.759ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "55018e4a-0ac2-40bb-b3bb-60e9720afe66", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
32782025-04-30T06:43:47.759ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
32792025-04-30T06:43:47.763ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f6e68841-385e-4294-94cb-cc24c5d2437d", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
32802025-04-30T06:43:47.763ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
32812025-04-30T06:43:47.769ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "382d5a93-0fbc-4e0c-91ff-4dde388cf0b5", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
32822025-04-30T06:43:47.769ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
32832025-04-30T06:43:47.789ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f860ada-1f81-46fe-b56c-e03d106a1318", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
32842025-04-30T06:43:47.789ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
32852025-04-30T06:43:47.792ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5012657d-a2cc-417f-a7eb-29526c02d1a2", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
32862025-04-30T06:43:47.792ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
32872025-04-30T06:43:47.793ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2578145e-88e1-4d07-a071-72809d77689f", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
32882025-04-30T06:43:47.793ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
32892025-04-30T06:43:47.794ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
32902025-04-30T06:43:47.794ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
32912025-04-30T06:43:47.794ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
32922025-04-30T06:43:47.794ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
32932025-04-30T06:43:47.795ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
32942025-04-30T06:43:47.808ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f282c6d1-9379-47a8-ad63-7a20d55fe35a", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
32952025-04-30T06:43:47.809ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
32962025-04-30T06:43:47.811ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32972025-04-30T06:43:47.817ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
32982025-04-30T06:43:47.818ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
32992025-04-30T06:43:47.818ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
33002025-04-30T06:43:47.818ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
33012025-04-30T06:43:47.831ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
33022025-04-30T06:43:47.832ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
33032025-04-30T06:43:47.833ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
33042025-04-30T06:43:47.837ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5ea9bb19-6998-4dee-9ad8-eb84e4b50e7c", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
33052025-04-30T06:43:47.838ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
33062025-04-30T06:43:47.846ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e8522969-2026-42d1-bef0-cc979d93ab25", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
33072025-04-30T06:43:47.846ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
33082025-04-30T06:43:47.856ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f1ecd99d-ac8c-4c7c-bf2b-686eaa019ffd", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
33092025-04-30T06:43:47.856ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
33102025-04-30T06:43:47.871ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9f5a5577-260d-425b-bc66-52b8c5bbe33a", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
33112025-04-30T06:43:47.871ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
33122025-04-30T06:43:47.885ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "07bf8271-08f1-467a-b267-ccfe16cbd3bc", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
33132025-04-30T06:43:47.885ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
33142025-04-30T06:43:47.899ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2c3d6bde-972d-4e27-b523-da24af99fd04", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
33152025-04-30T06:43:47.904ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
33162025-04-30T06:43:47.914ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
33172025-04-30T06:43:47.945ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4eff7578-caa7-4916-9aaa-95eb353cbb14", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
33182025-04-30T06:43:47.945ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
33192025-04-30T06:43:47.947ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "66bef7f1-2a7f-4ff5-952f-b9a09a0b787c", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
33202025-04-30T06:43:47.947ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
33212025-04-30T06:43:47.949ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "36d66ceb-9c3e-46a5-bb5d-4446d07d0184", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
33222025-04-30T06:43:47.949ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
33232025-04-30T06:43:47.953ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "62a7484e-c6aa-4b9f-9783-797e997fd169", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
33242025-04-30T06:43:47.953ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
33252025-04-30T06:43:47.955ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2c6183cf-5107-4cdb-a344-71a6a1e20405", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
33262025-04-30T06:43:47.956ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
33272025-04-30T06:43:47.958ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "80c445ee-8ac0-4f00-a8c1-96e5cce596dc", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
33282025-04-30T06:43:47.961ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
33292025-04-30T06:43:47.963ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "812ae7e4-58f2-44c0-adf8-d931d55f686d", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
33302025-04-30T06:43:47.963ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
33312025-04-30T06:43:47.965ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
33322025-04-30T06:43:47.965ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
33332025-04-30T06:43:47.965ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
33342025-04-30T06:43:47.965ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
33352025-04-30T06:43:47.966ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
33362025-04-30T06:43:47.967ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "aa88875d-6dc6-4054-8dd5-351233bff3de", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
33372025-04-30T06:43:47.967ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
33382025-04-30T06:43:47.968ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33392025-04-30T06:43:47.974ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
33402025-04-30T06:43:47.974ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
33412025-04-30T06:43:47.975ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
33422025-04-30T06:43:47.975ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
33432025-04-30T06:43:47.977ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
33442025-04-30T06:43:47.977ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
33452025-04-30T06:43:47.978ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
33462025-04-30T06:43:47.979ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "de94960e-dd17-41c7-89b2-8e53bd79f785", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
33472025-04-30T06:43:47.980ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
33482025-04-30T06:43:47.983ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e6506d1c-f215-43a5-8f45-2f846a99b0e4", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
33492025-04-30T06:43:47.983ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
33502025-04-30T06:43:47.991ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "66e277f4-f4bd-4179-a87d-c005b29544cb", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
33512025-04-30T06:43:47.992ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
33522025-04-30T06:43:47.999ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "49e17c13-1fb5-46ae-bf30-1fbbe5c14125", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
33532025-04-30T06:43:47.999ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
33542025-04-30T06:43:48.010ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4783f480-d34c-43c7-8cc7-628ec9bfb0f2", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
33552025-04-30T06:43:48.010ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
33562025-04-30T06:43:48.011ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d23fa363-59d1-43b6-a099-eadc06fbb969", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
33572025-04-30T06:43:48.012ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
33582025-04-30T06:43:48.013ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "050ca62d-8a23-45cd-8721-33d5db44a08c", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
33592025-04-30T06:43:48.013ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
33602025-04-30T06:43:48.014ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "de98b91f-68af-48b1-87e3-9d05c80a89f7", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
33612025-04-30T06:43:48.014ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
33622025-04-30T06:43:48.015ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
33632025-04-30T06:43:48.015ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
33642025-04-30T06:43:48.015ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
33652025-04-30T06:43:48.016ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
33662025-04-30T06:43:48.020ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33672025-04-30T06:43:48.024ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
33682025-04-30T06:43:48.024ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
33692025-04-30T06:43:48.024ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
33702025-04-30T06:43:48.024ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
33712025-04-30T06:43:48.024ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
33722025-04-30T06:43:48.031ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
33732025-04-30T06:43:48.040ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33742025-04-30T06:43:48.043ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
33752025-04-30T06:43:48.044ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
33762025-04-30T06:43:48.045ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
33772025-04-30T06:43:48.046ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
33782025-04-30T06:43:48.114ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
33792025-04-30T06:43:48.114ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory
33802025-04-30T06:43:48.117ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:52969 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
33812025-04-30T06:43:48.118ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:52969 req_id = 7413a306-ba96-4b02-93a7-732d331c1947 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
33822025-04-30T06:43:48.118ZINFOtest_unauthorized (dropshot): request completed latency_us = 691 local_addr = [::1]:57786 method = GET remote_addr = [::1]:52969 req_id = 7413a306-ba96-4b02-93a7-732d331c1947 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
33832025-04-30T06:43:48.119ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
33842025-04-30T06:43:48.122ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = http://[::1]:57786 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:57786/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "7413a306-ba96-4b02-93a7-732d331c1947", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
33852025-04-30T06:43:48.125ZDEBGtest_unauthorized (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
33862025-04-30T06:43:48.125ZDEBGtest_unauthorized (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
33872025-04-30T06:43:48.125ZDEBGtest_unauthorized (BackgroundTasks): finished collection background_task = inventory_collection
33882025-04-30T06:43:48.130ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
33892025-04-30T06:43:48.133ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
33902025-04-30T06:43:48.133ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
33912025-04-30T06:43:48.133ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
33922025-04-30T06:43:48.134ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
33932025-04-30T06:43:48.138ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33942025-04-30T06:43:48.144ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
33952025-04-30T06:43:48.145ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
33962025-04-30T06:43:48.146ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
33972025-04-30T06:43:48.147ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
33982025-04-30T06:43:48.174ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
33992025-04-30T06:43:48.174ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
34002025-04-30T06:43:48.176ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
34012025-04-30T06:43:48.176ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
34022025-04-30T06:43:48.176ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
34032025-04-30T06:43:48.177ZTRACtest_unauthorized (results-sink): batch interval expired, but no samples to insert collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
34042025-04-30T06:43:48.177ZTRACtest_unauthorized (results-sink-cluster): batch interval expired, but no samples to insert collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
34052025-04-30T06:43:48.180ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34062025-04-30T06:43:48.188ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
34072025-04-30T06:43:48.188ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
34082025-04-30T06:43:48.188ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
34092025-04-30T06:43:48.190ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
34102025-04-30T06:43:48.268ZWARNtest_unauthorized (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
34112025-04-30T06:43:48.269ZWARNtest_unauthorized (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
34122025-04-30T06:43:48.270ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 723.734794ms iteration = 4 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
34132025-04-30T06:43:48.270ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 5 reason = Signaled
34142025-04-30T06:43:48.271ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
34152025-04-30T06:43:48.272ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
34162025-04-30T06:43:48.273ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
34172025-04-30T06:43:48.274ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
34182025-04-30T06:43:48.276ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
34192025-04-30T06:43:48.278ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
34202025-04-30T06:43:48.280ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
34212025-04-30T06:43:48.280ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
34222025-04-30T06:43:48.287ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34232025-04-30T06:43:48.293ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
34242025-04-30T06:43:48.300ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
34252025-04-30T06:43:48.301ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
34262025-04-30T06:43:48.302ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
34272025-04-30T06:43:48.303ZDEBGtest_unauthorized (producer-registration-task): will register with Nexus at 127.0.0.1:34029
34282025-04-30T06:43:48.384ZDEBGtest_unauthorized (producer-registration-task): client request body = Some(Body) method = POST uri = http://127.0.0.1:34029/metrics/producers
34292025-04-30T06:43:48.384ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1a8b2318-abf8-406d-a389-112ab91a0de8, 0)]
34302025-04-30T06:43:48.384ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
34312025-04-30T06:43:48.385ZINFOtest_unauthorized (dropshot_internal): accepted connection local_addr = 127.0.0.1:34029 remote_addr = 127.0.0.1:38441
34322025-04-30T06:43:48.386ZTRACtest_unauthorized (dropshot_internal): incoming request local_addr = 127.0.0.1:34029 method = POST remote_addr = 127.0.0.1:38441 req_id = bd7887c6-3c63-4bfa-ab0b-6e359ef6d6fd uri = /metrics/producers
34332025-04-30T06:43:48.386ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: b8 fe 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:51831 req_id = 3b7f48a5-57df-4aa9-a154-c0545388ce2a
34342025-04-30T06:43:48.386ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 47358,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51831 req_id = 3b7f48a5-57df-4aa9-a154-c0545388ce2a
34352025-04-30T06:43:48.386ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
34362025-04-30T06:43:48.386ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
34372025-04-30T06:43:48.386ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:51831 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 3b7f48a5-57df-4aa9-a154-c0545388ce2a
34382025-04-30T06:43:48.387ZTRACtest_unauthorized (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = POST remote_addr = 127.0.0.1:38441 req_id = bd7887c6-3c63-4bfa-ab0b-6e359ef6d6fd resource = Database uri = /metrics/producers
34392025-04-30T06:43:48.387ZDEBGtest_unauthorized (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = POST remote_addr = 127.0.0.1:38441 req_id = bd7887c6-3c63-4bfa-ab0b-6e359ef6d6fd roles = RoleSet { roles: {} } uri = /metrics/producers
34402025-04-30T06:43:48.387ZDEBGtest_unauthorized (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = POST remote_addr = 127.0.0.1:38441 req_id = bd7887c6-3c63-4bfa-ab0b-6e359ef6d6fd resource = Database result = Ok(()) uri = /metrics/producers
34412025-04-30T06:43:48.388ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4293, tv_nsec: 187358472 } })
34422025-04-30T06:43:48.388ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
34432025-04-30T06:43:48.389ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 26 4d 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e &M..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:55557 req_id = a8d2a61a-0591-47cc-836f-c0873ecf4aa7
34442025-04-30T06:43:48.389ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 9805,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:55557 req_id = a8d2a61a-0591-47cc-836f-c0873ecf4aa7
34452025-04-30T06:43:48.389ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
34462025-04-30T06:43:48.389ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:55557 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = a8d2a61a-0591-47cc-836f-c0873ecf4aa7
34472025-04-30T06:43:48.440ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
34482025-04-30T06:43:48.442ZINFOtest_unauthorized (nexus): assigned collector to new producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 producer_id = 8d752adb-08b9-405f-a11b-66019bf447b5
34492025-04-30T06:43:48.442ZINFOtest_unauthorized (dropshot_internal): request completed latency_us = 56328 local_addr = 127.0.0.1:34029 method = POST remote_addr = 127.0.0.1:38441 req_id = bd7887c6-3c63-4bfa-ab0b-6e359ef6d6fd response_code = 201 uri = /metrics/producers
34502025-04-30T06:43:48.443ZDEBGtest_unauthorized (producer-registration-task): client response result = Ok(Response { url: "http://127.0.0.1:34029/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "bd7887c6-3c63-4bfa-ab0b-6e359ef6d6fd", "content-length": "41", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
34512025-04-30T06:43:48.446ZDEBGtest_unauthorized (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s wait_period = 150s
34522025-04-30T06:43:48.449ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: fa 96 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:59029 req_id = c3aa3f7d-802a-452a-b540-5450ed1fb3db
34532025-04-30T06:43:48.449ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 64150,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:59029 req_id = c3aa3f7d-802a-452a-b540-5450ed1fb3db
34542025-04-30T06:43:48.449ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
34552025-04-30T06:43:48.449ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 64150, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:59029 req_id = c3aa3f7d-802a-452a-b540-5450ed1fb3db
34562025-04-30T06:43:48.450ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
34572025-04-30T06:43:48.450ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
34582025-04-30T06:43:48.450ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
34592025-04-30T06:43:48.452ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "0b0f7e5c-6e0b-40cd-9c2b-276a90aec442", "content-length": "835", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
34602025-04-30T06:43:48.452ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
34612025-04-30T06:43:48.454ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d8a6a69d-4327-4b9a-ab1f-f5bdafbf3fe5", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
34622025-04-30T06:43:48.454ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
34632025-04-30T06:43:48.470ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "14b0e65d-7d0d-4f17-a22f-27abb6257959", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
34642025-04-30T06:43:48.471ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
34652025-04-30T06:43:48.475ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d2cd5b20-c870-4601-b453-3c9a238d933e", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
34662025-04-30T06:43:48.475ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
34672025-04-30T06:43:48.482ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fed50867-7871-4dc7-869d-f54d333e5b17", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
34682025-04-30T06:43:48.482ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
34692025-04-30T06:43:48.491ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
34702025-04-30T06:43:48.494ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0a24b8bf-d25b-4612-97ab-91db2e2a91e8", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
34712025-04-30T06:43:48.494ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
34722025-04-30T06:43:48.503ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "58445fd9-06f2-4837-9e70-7f0fbce6f802", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
34732025-04-30T06:43:48.503ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
34742025-04-30T06:43:48.505ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1c043c9f-dc4b-4a80-9d12-17db468744d5", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
34752025-04-30T06:43:48.505ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
34762025-04-30T06:43:48.506ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ff1b5e47-4488-4f25-9f9f-e5422e313037", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
34772025-04-30T06:43:48.507ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
34782025-04-30T06:43:48.509ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1f6a2bbe-21dd-4133-9a5c-23adca78c93a", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
34792025-04-30T06:43:48.509ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
34802025-04-30T06:43:48.512ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c23ca058-ca28-4f3a-801e-f870657bb66f", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
34812025-04-30T06:43:48.512ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
34822025-04-30T06:43:48.514ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "587561b0-2c48-4209-bed5-802119206720", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
34832025-04-30T06:43:48.515ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
34842025-04-30T06:43:48.544ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c8b9700c-9125-4cb2-be98-1e05a0ef6431", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
34852025-04-30T06:43:48.545ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
34862025-04-30T06:43:48.545ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
34872025-04-30T06:43:48.545ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
34882025-04-30T06:43:48.545ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
34892025-04-30T06:43:48.545ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
34902025-04-30T06:43:48.546ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
34912025-04-30T06:43:48.555ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34922025-04-30T06:43:48.561ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
34932025-04-30T06:43:48.561ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
34942025-04-30T06:43:48.561ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
34952025-04-30T06:43:48.562ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
34962025-04-30T06:43:48.580ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
34972025-04-30T06:43:48.580ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
34982025-04-30T06:43:48.581ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
34992025-04-30T06:43:48.601ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2338170b-ea80-4174-a84c-99993a15ba40", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
35002025-04-30T06:43:48.601ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
35012025-04-30T06:43:48.644ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "56bf63c4-72a0-494e-b8f6-042c7aec22e3", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
35022025-04-30T06:43:48.644ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
35032025-04-30T06:43:48.649ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "db1dbe57-4446-4904-9fa4-9fe77262f849", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
35042025-04-30T06:43:48.650ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
35052025-04-30T06:43:48.656ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
35062025-04-30T06:43:48.660ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "210745ba-fac1-4719-bc57-431a2bfd49da", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
35072025-04-30T06:43:48.661ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
35082025-04-30T06:43:48.665ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f18468f-6150-4a68-a897-b3368e17de3e", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
35092025-04-30T06:43:48.665ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
35102025-04-30T06:43:48.668ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ac016d89-5fea-4ab3-9074-a1fc355d4fff", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
35112025-04-30T06:43:48.668ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
35122025-04-30T06:43:48.669ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "90283aad-953b-4458-9945-e5f9da23335e", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
35132025-04-30T06:43:48.670ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
35142025-04-30T06:43:48.671ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "51c32773-ba8a-4fd7-b38a-9a3a5f6e7448", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
35152025-04-30T06:43:48.672ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
35162025-04-30T06:43:48.676ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "351fcce5-4d09-4f6b-8ee0-a0cad3069e3b", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
35172025-04-30T06:43:48.676ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
35182025-04-30T06:43:48.678ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "085e6a4f-2544-40de-8dd7-3c160fcf36f9", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
35192025-04-30T06:43:48.679ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
35202025-04-30T06:43:48.682ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0e2a6ab5-b805-4ba1-a11b-659df991e3b9", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
35212025-04-30T06:43:48.682ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
35222025-04-30T06:43:48.688ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "20a95e0b-e8b4-4a49-bf6f-24793147d7e3", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
35232025-04-30T06:43:48.688ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
35242025-04-30T06:43:48.694ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "97a3bc6f-4730-476e-97d8-9bcd1a179eaa", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
35252025-04-30T06:43:48.694ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
35262025-04-30T06:43:48.696ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8088041b-e9f6-44d1-a9bd-f86528616204", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
35272025-04-30T06:43:48.696ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
35282025-04-30T06:43:48.699ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a86d73f8-a646-4f2b-860e-ea95ec6374d6", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
35292025-04-30T06:43:48.699ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
35302025-04-30T06:43:48.701ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ea34e99d-8d2f-48ca-aec7-7412bc95aeac", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
35312025-04-30T06:43:48.701ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
35322025-04-30T06:43:48.704ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "26ab0170-bcd2-4edd-97f8-278119ca7445", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
35332025-04-30T06:43:48.704ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
35342025-04-30T06:43:48.706ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ab0a24ed-b11a-4a0d-8a99-c344f22d2da1", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
35352025-04-30T06:43:48.707ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
35362025-04-30T06:43:48.707ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
35372025-04-30T06:43:48.707ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
35382025-04-30T06:43:48.707ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
35392025-04-30T06:43:48.707ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
35402025-04-30T06:43:48.708ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
35412025-04-30T06:43:48.710ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "939c6ace-c04f-474d-85d1-2b10ee1f9be6", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
35422025-04-30T06:43:48.710ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
35432025-04-30T06:43:48.711ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35442025-04-30T06:43:48.716ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
35452025-04-30T06:43:48.716ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
35462025-04-30T06:43:48.716ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
35472025-04-30T06:43:48.717ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
35482025-04-30T06:43:48.717ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c04b22c2-b8ab-4aaa-aac3-4b8dc14484ef", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
35492025-04-30T06:43:48.717ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
35502025-04-30T06:43:48.719ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f05d8fa-bc1e-4bb4-b6ed-3887b94f7dd7", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
35512025-04-30T06:43:48.720ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
35522025-04-30T06:43:48.720ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
35532025-04-30T06:43:48.720ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
35542025-04-30T06:43:48.722ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
35552025-04-30T06:43:48.724ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4e643f7b-f6b9-4565-93a7-0d7dd5c3041e", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
35562025-04-30T06:43:48.725ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
35572025-04-30T06:43:48.728ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8d319e61-751b-4e07-b229-ccf2a3917a26", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
35582025-04-30T06:43:48.729ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
35592025-04-30T06:43:48.733ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5c737c8a-4ec1-4a7d-8ebe-d093af7943e0", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
35602025-04-30T06:43:48.733ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
35612025-04-30T06:43:48.737ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0fe0a2dc-6513-46f8-b65f-78c3a609d4f3", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
35622025-04-30T06:43:48.737ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
35632025-04-30T06:43:48.742ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6ff2a450-0d72-4dab-b9d7-8a13c7a2b466", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
35642025-04-30T06:43:48.743ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
35652025-04-30T06:43:48.748ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5f3ef474-854e-42b3-95df-e549050ca305", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
35662025-04-30T06:43:48.748ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
35672025-04-30T06:43:48.752ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
35682025-04-30T06:43:48.754ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d88c3b4f-e873-4037-8883-ce9c0206b758", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
35692025-04-30T06:43:48.754ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
35702025-04-30T06:43:48.756ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2f057bbf-9218-48ce-9991-e5373de81fd9", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
35712025-04-30T06:43:48.756ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
35722025-04-30T06:43:48.757ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b3aa086a-b608-4fce-a1c2-b2568fa150dd", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
35732025-04-30T06:43:48.758ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
35742025-04-30T06:43:48.759ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9f18adbb-ecf8-4dac-82cf-450303d672cb", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
35752025-04-30T06:43:48.759ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
35762025-04-30T06:43:48.760ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "09cf3399-f740-4bf2-9c83-04021e965dc0", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
35772025-04-30T06:43:48.760ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
35782025-04-30T06:43:48.760ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
35792025-04-30T06:43:48.760ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
35802025-04-30T06:43:48.760ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
35812025-04-30T06:43:48.762ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
35822025-04-30T06:43:48.766ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35832025-04-30T06:43:48.770ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
35842025-04-30T06:43:48.770ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
35852025-04-30T06:43:48.770ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
35862025-04-30T06:43:48.770ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
35872025-04-30T06:43:48.770ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
35882025-04-30T06:43:48.770ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
35892025-04-30T06:43:48.773ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35902025-04-30T06:43:48.779ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
35912025-04-30T06:43:48.779ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
35922025-04-30T06:43:48.779ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
35932025-04-30T06:43:48.779ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
35942025-04-30T06:43:48.861ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
35952025-04-30T06:43:48.861ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory
35962025-04-30T06:43:48.862ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
35972025-04-30T06:43:48.862ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
35982025-04-30T06:43:48.863ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
35992025-04-30T06:43:48.863ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
36002025-04-30T06:43:48.865ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
36012025-04-30T06:43:48.866ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:42030 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
36022025-04-30T06:43:48.869ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:42030 req_id = e7e34120-31ec-4ecb-9b60-93172f376a88 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
36032025-04-30T06:43:48.872ZINFOtest_unauthorized (dropshot): request completed latency_us = 2675 local_addr = [::1]:57786 method = GET remote_addr = [::1]:42030 req_id = e7e34120-31ec-4ecb-9b60-93172f376a88 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
36042025-04-30T06:43:48.873ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36052025-04-30T06:43:48.878ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
36062025-04-30T06:43:48.879ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
36072025-04-30T06:43:48.879ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
36082025-04-30T06:43:48.880ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
36092025-04-30T06:43:48.880ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = http://[::1]:57786 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:57786/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "e7e34120-31ec-4ecb-9b60-93172f376a88", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
36102025-04-30T06:43:48.886ZDEBGtest_unauthorized (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
36112025-04-30T06:43:48.886ZDEBGtest_unauthorized (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
36122025-04-30T06:43:48.886ZDEBGtest_unauthorized (BackgroundTasks): finished collection background_task = inventory_collection
36132025-04-30T06:43:48.888ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
36142025-04-30T06:43:48.888ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
36152025-04-30T06:43:48.888ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
36162025-04-30T06:43:48.888ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
36172025-04-30T06:43:48.889ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
36182025-04-30T06:43:48.894ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
36192025-04-30T06:43:48.894ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
36202025-04-30T06:43:48.894ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
36212025-04-30T06:43:48.896ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36222025-04-30T06:43:48.903ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
36232025-04-30T06:43:48.904ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
36242025-04-30T06:43:48.904ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
36252025-04-30T06:43:48.905ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
36262025-04-30T06:43:48.979ZWARNtest_unauthorized (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
36272025-04-30T06:43:48.979ZWARNtest_unauthorized (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
36282025-04-30T06:43:48.979ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 709.126638ms iteration = 5 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
36292025-04-30T06:43:48.979ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 6 reason = Signaled
36302025-04-30T06:43:48.979ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
36312025-04-30T06:43:48.979ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
36322025-04-30T06:43:48.979ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
36332025-04-30T06:43:48.980ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
36342025-04-30T06:43:48.980ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
36352025-04-30T06:43:48.985ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36362025-04-30T06:43:48.989ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
36372025-04-30T06:43:48.989ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
36382025-04-30T06:43:48.989ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
36392025-04-30T06:43:48.990ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
36402025-04-30T06:43:48.990ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
36412025-04-30T06:43:48.993ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1a8b2318-abf8-406d-a389-112ab91a0de8, 0)]
36422025-04-30T06:43:48.993ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
36432025-04-30T06:43:48.994ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 41 9a 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 A............_mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:56329 req_id = d4405b61-610e-40a3-a8a5-49db06873322
36442025-04-30T06:43:48.994ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 16794,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:56329 req_id = d4405b61-610e-40a3-a8a5-49db06873322
36452025-04-30T06:43:48.994ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
36462025-04-30T06:43:48.995ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
36472025-04-30T06:43:48.995ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:56329 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = d4405b61-610e-40a3-a8a5-49db06873322
36482025-04-30T06:43:48.996ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4293, tv_nsec: 795088103 } })
36492025-04-30T06:43:48.996ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
36502025-04-30T06:43:48.997ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: bc 68 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .h..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:55229 req_id = ee30dcd4-0bf5-417e-9dc7-964d38c9eba9
36512025-04-30T06:43:48.998ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 48232,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:55229 req_id = ee30dcd4-0bf5-417e-9dc7-964d38c9eba9
36522025-04-30T06:43:48.999ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
36532025-04-30T06:43:48.999ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:55229 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = ee30dcd4-0bf5-417e-9dc7-964d38c9eba9
36542025-04-30T06:43:49.070ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
36552025-04-30T06:43:49.071ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
36562025-04-30T06:43:49.071ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
36572025-04-30T06:43:49.071ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
36582025-04-30T06:43:49.071ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
36592025-04-30T06:43:49.072ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
36602025-04-30T06:43:49.072ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: c1 e0 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:60640 req_id = 98f11e1b-b55a-4ef8-803d-cdb9a86ec59a
36612025-04-30T06:43:49.072ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 49632,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60640 req_id = 98f11e1b-b55a-4ef8-803d-cdb9a86ec59a
36622025-04-30T06:43:49.072ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
36632025-04-30T06:43:49.073ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 49632, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:60640 req_id = 98f11e1b-b55a-4ef8-803d-cdb9a86ec59a
36642025-04-30T06:43:49.073ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
36652025-04-30T06:43:49.073ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
36662025-04-30T06:43:49.073ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
36672025-04-30T06:43:49.075ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36682025-04-30T06:43:49.079ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
36692025-04-30T06:43:49.079ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
36702025-04-30T06:43:49.079ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
36712025-04-30T06:43:49.080ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
36722025-04-30T06:43:49.080ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "05b9a53e-d549-4b1b-9f93-5eb8b15933e3", "content-length": "835", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
36732025-04-30T06:43:49.081ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
36742025-04-30T06:43:49.083ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6bc44bde-bc17-47b6-b8e9-2320fa079ba6", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
36752025-04-30T06:43:49.083ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
36762025-04-30T06:43:49.084ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
36772025-04-30T06:43:49.085ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
36782025-04-30T06:43:49.086ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
36792025-04-30T06:43:49.091ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0c68ffe5-f5c5-4f55-b352-2577035eb922", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
36802025-04-30T06:43:49.091ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
36812025-04-30T06:43:49.097ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2346be00-97f1-4513-87e0-bc5f85eb9ce1", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
36822025-04-30T06:43:49.097ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
36832025-04-30T06:43:49.099ZINFOtest_unauthorized (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
36842025-04-30T06:43:49.099ZDEBGtest_unauthorized (oximeter): client request body = None method = GET uri = http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
36852025-04-30T06:43:49.100ZTRACtest_unauthorized (dropshot_internal): incoming request local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 6a32c548-cb78-4c44-b70b-7f17acedfbf5 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
36862025-04-30T06:43:49.100ZTRACtest_unauthorized (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 6a32c548-cb78-4c44-b70b-7f17acedfbf5 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
36872025-04-30T06:43:49.100ZDEBGtest_unauthorized (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 6a32c548-cb78-4c44-b70b-7f17acedfbf5 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
36882025-04-30T06:43:49.101ZDEBGtest_unauthorized (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 6a32c548-cb78-4c44-b70b-7f17acedfbf5 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
36892025-04-30T06:43:49.106ZINFOtest_unauthorized (dropshot_internal): request completed latency_us = 6209 local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 6a32c548-cb78-4c44-b70b-7f17acedfbf5 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
36902025-04-30T06:43:49.107ZDEBGtest_unauthorized (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "6a32c548-cb78-4c44-b70b-7f17acedfbf5", "content-length": "405", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
36912025-04-30T06:43:49.107ZDEBGtest_unauthorized (oximeter): client request body = None method = GET uri = http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
36922025-04-30T06:43:49.108ZTRACtest_unauthorized (dropshot_internal): incoming request local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 141ce4b6-dd1a-49e6-ab34-4fa7ab2c0dff uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
36932025-04-30T06:43:49.108ZTRACtest_unauthorized (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 141ce4b6-dd1a-49e6-ab34-4fa7ab2c0dff resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
36942025-04-30T06:43:49.108ZDEBGtest_unauthorized (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 141ce4b6-dd1a-49e6-ab34-4fa7ab2c0dff roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
36952025-04-30T06:43:49.109ZDEBGtest_unauthorized (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 141ce4b6-dd1a-49e6-ab34-4fa7ab2c0dff resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
36962025-04-30T06:43:49.109ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a72caf4-73b6-4d23-9803-779925cbde38", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
36972025-04-30T06:43:49.110ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
36982025-04-30T06:43:49.114ZINFOtest_unauthorized (dropshot_internal): request completed latency_us = 5807 local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 141ce4b6-dd1a-49e6-ab34-4fa7ab2c0dff response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
36992025-04-30T06:43:49.114ZDEBGtest_unauthorized (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "141ce4b6-dd1a-49e6-ab34-4fa7ab2c0dff", "content-length": "29", "date": "Wed, 30 Apr 2025 06:43:48 GMT"} })
37002025-04-30T06:43:49.114ZDEBGtest_unauthorized (oximeter-agent): registered new metric producer address = 127.0.0.1:53884 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 8d752adb-08b9-405f-a11b-66019bf447b5
37012025-04-30T06:43:49.197ZTRACtest_unauthorized (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 8d752adb-08b9-405f-a11b-66019bf447b5
37022025-04-30T06:43:49.198ZTRACtest_unauthorized (results-sink): batch interval expired, but no samples to insert collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
37032025-04-30T06:43:49.199ZTRACtest_unauthorized (results-sink-cluster): batch interval expired, but no samples to insert collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
37042025-04-30T06:43:49.199ZDEBGtest_unauthorized (oximeter-agent): registered new metric producer address = [::1]:50005 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
37052025-04-30T06:43:49.200ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9a31529f-cbff-4227-9c7b-4dd2668677fd", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
37062025-04-30T06:43:49.200ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
37072025-04-30T06:43:49.283ZTRACtest_unauthorized (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
37082025-04-30T06:43:49.290ZINFOtest_unauthorized (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
37092025-04-30T06:43:49.291ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
37102025-04-30T06:43:49.294ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a46bb393-6f86-428a-ac0c-1e2a2bb0f240", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
37112025-04-30T06:43:49.294ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
37122025-04-30T06:43:49.298ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4c6a92be-7a6b-4379-b7b4-276139adc5df", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
37132025-04-30T06:43:49.298ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
37142025-04-30T06:43:49.300ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "69b8bcb9-8e59-4fb8-bfd1-ac825d07ce19", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
37152025-04-30T06:43:49.300ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
37162025-04-30T06:43:49.301ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1279c738-be20-4c43-9d95-0041181bd9bd", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
37172025-04-30T06:43:49.302ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
37182025-04-30T06:43:49.303ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "19b40259-4662-441f-80ae-e5e563add5c1", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
37192025-04-30T06:43:49.304ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
37202025-04-30T06:43:49.304ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f60a7d25-b519-4611-8179-96e031ebceb4", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
37212025-04-30T06:43:49.305ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
37222025-04-30T06:43:49.307ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc3a94e6-0d16-4af7-b6f1-56d53798b813", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
37232025-04-30T06:43:49.307ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
37242025-04-30T06:43:49.312ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3033e080-21c2-4112-a98b-59008efb6778", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
37252025-04-30T06:43:49.312ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
37262025-04-30T06:43:49.319ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "48a73e0e-2db6-4498-91b9-28361051d726", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
37272025-04-30T06:43:49.319ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
37282025-04-30T06:43:49.321ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8fd93a8b-85fd-4eb4-9265-30c2821f4e29", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
37292025-04-30T06:43:49.322ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
37302025-04-30T06:43:49.325ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5ea2b9c8-bf95-49d7-bf3b-c9ada8a2b436", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
37312025-04-30T06:43:49.325ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
37322025-04-30T06:43:49.328ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "27bb67b1-00ba-47a6-b7e4-aef2fa606e03", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
37332025-04-30T06:43:49.328ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
37342025-04-30T06:43:49.331ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa484fcd-fa25-49bc-a9ea-dad1edd9c656", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
37352025-04-30T06:43:49.332ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
37362025-04-30T06:43:49.333ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9e1b78b2-924e-4dfe-952a-9414c419585c", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
37372025-04-30T06:43:49.333ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
37382025-04-30T06:43:49.334ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b9c4e75f-56db-48e4-9920-973c57a80492", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
37392025-04-30T06:43:49.334ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
37402025-04-30T06:43:49.335ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8b8f406b-bfa7-4830-aee6-ea1768da712c", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
37412025-04-30T06:43:49.335ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
37422025-04-30T06:43:49.336ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "418cfe5a-c6bf-4875-aa47-4578f77ffff7", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
37432025-04-30T06:43:49.336ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
37442025-04-30T06:43:49.337ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "02485c0e-e721-4ce2-9482-2cdea500eb61", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
37452025-04-30T06:43:49.337ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
37462025-04-30T06:43:49.341ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e068e20f-63a1-4feb-b1bf-0659e6f40e6d", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
37472025-04-30T06:43:49.341ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
37482025-04-30T06:43:49.342ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
37492025-04-30T06:43:49.342ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
37502025-04-30T06:43:49.342ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
37512025-04-30T06:43:49.342ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
37522025-04-30T06:43:49.344ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
37532025-04-30T06:43:49.346ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "106a5fa3-c610-4560-a427-3c7751c0e6d7", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
37542025-04-30T06:43:49.346ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
37552025-04-30T06:43:49.347ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37562025-04-30T06:43:49.364ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
37572025-04-30T06:43:49.364ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
37582025-04-30T06:43:49.364ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
37592025-04-30T06:43:49.364ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
37602025-04-30T06:43:49.366ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "754b4d38-169c-4671-8745-5d6ecb1be92a", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
37612025-04-30T06:43:49.366ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
37622025-04-30T06:43:49.367ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
37632025-04-30T06:43:49.367ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
37642025-04-30T06:43:49.367ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
37652025-04-30T06:43:49.369ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ae749b6d-0f12-43bc-9e84-5662ca6a0253", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
37662025-04-30T06:43:49.369ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
37672025-04-30T06:43:49.377ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9fbe1998-78b3-466b-865d-d56912b8eb64", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
37682025-04-30T06:43:49.377ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
37692025-04-30T06:43:49.377ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e49357de-d39e-4c5a-a16d-40bea2524e1e", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
37702025-04-30T06:43:49.377ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
37712025-04-30T06:43:49.377ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d224bcc0-4b89-423e-ae4e-f37246fc4f3d", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
37722025-04-30T06:43:49.377ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
37732025-04-30T06:43:49.377ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "eacef95c-cc9f-46b4-a8cb-b14b7fc43f39", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
37742025-04-30T06:43:49.377ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
37752025-04-30T06:43:49.377ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2f7443d4-f760-424c-b3af-d6a3ac618e2e", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
37762025-04-30T06:43:49.377ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
37772025-04-30T06:43:49.377ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "22cb7e2b-ad2b-4838-bff3-ced28dbc5eef", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
37782025-04-30T06:43:49.377ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
37792025-04-30T06:43:49.379ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a2027c29-6dde-4780-bc7c-d6d66327a8fc", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
37802025-04-30T06:43:49.379ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
37812025-04-30T06:43:49.381ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2a2cc4ae-73b2-42bf-a372-9543c0eb170a", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
37822025-04-30T06:43:49.381ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
37832025-04-30T06:43:49.384ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b9961b56-fdef-4b05-aa59-826792b25d01", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
37842025-04-30T06:43:49.384ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
37852025-04-30T06:43:49.388ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7f9a1657-5384-4b1c-a692-a392c10e79de", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
37862025-04-30T06:43:49.389ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
37872025-04-30T06:43:49.391ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "055f4b29-f420-459f-9e86-f651aeec78e5", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
37882025-04-30T06:43:49.391ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
37892025-04-30T06:43:49.394ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a946dde8-1bf6-452c-b1c3-9a4335f68b91", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
37902025-04-30T06:43:49.394ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
37912025-04-30T06:43:49.395ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7fea0029-f0ff-482d-a42f-4ca64835ce22", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
37922025-04-30T06:43:49.395ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
37932025-04-30T06:43:49.396ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f9d4765c-798b-42fc-95d3-fbc35bceac67", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
37942025-04-30T06:43:49.398ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
37952025-04-30T06:43:49.399ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "caa1e149-6eeb-4dae-86cc-99e2950e0446", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
37962025-04-30T06:43:49.399ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
37972025-04-30T06:43:49.402ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "dbcfc16b-484f-43bd-8af3-5fa38b95575e", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
37982025-04-30T06:43:49.402ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
37992025-04-30T06:43:49.403ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
38002025-04-30T06:43:49.404ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "359ef247-0fb0-428e-9edf-2f48d588480d", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
38012025-04-30T06:43:49.404ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
38022025-04-30T06:43:49.404ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
38032025-04-30T06:43:49.404ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
38042025-04-30T06:43:49.405ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
38052025-04-30T06:43:49.405ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
38062025-04-30T06:43:49.407ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38072025-04-30T06:43:49.412ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
38082025-04-30T06:43:49.413ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
38092025-04-30T06:43:49.413ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
38102025-04-30T06:43:49.413ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
38112025-04-30T06:43:49.413ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
38122025-04-30T06:43:49.414ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
38132025-04-30T06:43:49.419ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38142025-04-30T06:43:49.426ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
38152025-04-30T06:43:49.426ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
38162025-04-30T06:43:49.426ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
38172025-04-30T06:43:49.426ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
38182025-04-30T06:43:49.515ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
38192025-04-30T06:43:49.515ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory
38202025-04-30T06:43:49.516ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
38212025-04-30T06:43:49.516ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
38222025-04-30T06:43:49.516ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
38232025-04-30T06:43:49.516ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
38242025-04-30T06:43:49.517ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
38252025-04-30T06:43:49.517ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:36869 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
38262025-04-30T06:43:49.518ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:36869 req_id = fbaec815-8330-4f37-8912-dda4c91655b3 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
38272025-04-30T06:43:49.518ZINFOtest_unauthorized (dropshot): request completed latency_us = 435 local_addr = [::1]:57786 method = GET remote_addr = [::1]:36869 req_id = fbaec815-8330-4f37-8912-dda4c91655b3 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
38282025-04-30T06:43:49.518ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = http://[::1]:57786 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:57786/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "fbaec815-8330-4f37-8912-dda4c91655b3", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
38292025-04-30T06:43:49.520ZDEBGtest_unauthorized (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
38302025-04-30T06:43:49.520ZDEBGtest_unauthorized (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
38312025-04-30T06:43:49.520ZDEBGtest_unauthorized (BackgroundTasks): finished collection background_task = inventory_collection
38322025-04-30T06:43:49.522ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
38332025-04-30T06:43:49.523ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
38342025-04-30T06:43:49.523ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
38352025-04-30T06:43:49.523ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
38362025-04-30T06:43:49.523ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
38372025-04-30T06:43:49.526ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38382025-04-30T06:43:49.535ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
38392025-04-30T06:43:49.535ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
38402025-04-30T06:43:49.535ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
38412025-04-30T06:43:49.535ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
38422025-04-30T06:43:49.536ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38432025-04-30T06:43:49.539ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
38442025-04-30T06:43:49.540ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
38452025-04-30T06:43:49.540ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
38462025-04-30T06:43:49.540ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
38472025-04-30T06:43:49.544ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
38482025-04-30T06:43:49.544ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
38492025-04-30T06:43:49.545ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
38502025-04-30T06:43:49.639ZWARNtest_unauthorized (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
38512025-04-30T06:43:49.640ZWARNtest_unauthorized (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
38522025-04-30T06:43:49.640ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 660.435665ms iteration = 6 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
38532025-04-30T06:43:49.640ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 7 reason = Signaled
38542025-04-30T06:43:49.640ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
38552025-04-30T06:43:49.640ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
38562025-04-30T06:43:49.640ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
38572025-04-30T06:43:49.640ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
38582025-04-30T06:43:49.669ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
38592025-04-30T06:43:49.674ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38602025-04-30T06:43:49.711ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
38612025-04-30T06:43:49.711ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
38622025-04-30T06:43:49.711ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
38632025-04-30T06:43:49.712ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
38642025-04-30T06:43:49.713ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
38652025-04-30T06:43:49.732ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1a8b2318-abf8-406d-a389-112ab91a0de8, 0)]
38662025-04-30T06:43:49.733ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
38672025-04-30T06:43:49.734ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: a4 ae 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:51657 req_id = 6202174c-a2a5-45fa-b837-6ae64dd519a8
38682025-04-30T06:43:49.734ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 42158,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51657 req_id = 6202174c-a2a5-45fa-b837-6ae64dd519a8
38692025-04-30T06:43:49.735ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
38702025-04-30T06:43:49.735ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
38712025-04-30T06:43:49.735ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:51657 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 6202174c-a2a5-45fa-b837-6ae64dd519a8
38722025-04-30T06:43:49.736ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4294, tv_nsec: 535253094 } })
38732025-04-30T06:43:49.736ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
38742025-04-30T06:43:49.736ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 46 31 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e F1..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:59973 req_id = 0615957b-1e6f-4d66-9c77-1ea8ebbd72b3
38752025-04-30T06:43:49.736ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 17969,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:59973 req_id = 0615957b-1e6f-4d66-9c77-1ea8ebbd72b3
38762025-04-30T06:43:49.737ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
38772025-04-30T06:43:49.737ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:59973 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 0615957b-1e6f-4d66-9c77-1ea8ebbd72b3
38782025-04-30T06:43:49.808ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
38792025-04-30T06:43:49.809ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
38802025-04-30T06:43:49.809ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
38812025-04-30T06:43:49.809ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
38822025-04-30T06:43:49.809ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
38832025-04-30T06:43:49.810ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
38842025-04-30T06:43:49.810ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: f8 7b 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c .{..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:57362 req_id = 493dbef8-566d-41fb-b448-500cf7c5f35c
38852025-04-30T06:43:49.810ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 63611,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:57362 req_id = 493dbef8-566d-41fb-b448-500cf7c5f35c
38862025-04-30T06:43:49.811ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
38872025-04-30T06:43:49.811ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 63611, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:57362 req_id = 493dbef8-566d-41fb-b448-500cf7c5f35c
38882025-04-30T06:43:49.811ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
38892025-04-30T06:43:49.811ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
38902025-04-30T06:43:49.811ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
38912025-04-30T06:43:49.815ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38922025-04-30T06:43:49.822ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
38932025-04-30T06:43:49.829ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
38942025-04-30T06:43:49.829ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
38952025-04-30T06:43:49.830ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
38962025-04-30T06:43:49.833ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
38972025-04-30T06:43:49.833ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
38982025-04-30T06:43:49.833ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
38992025-04-30T06:43:49.834ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "ce42b95e-8b05-4ea9-9b4e-9e96668a98fe", "content-length": "835", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
39002025-04-30T06:43:49.834ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
39012025-04-30T06:43:49.837ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5650ca03-7a92-4268-b930-e4d3faa94c7f", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
39022025-04-30T06:43:49.837ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
39032025-04-30T06:43:49.848ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "10d4b6d8-e4bb-422e-baf3-eeb330d19158", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
39042025-04-30T06:43:49.848ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
39052025-04-30T06:43:49.864ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e8072104-f6ef-4180-86bf-0f49b09c2c54", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
39062025-04-30T06:43:49.864ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
39072025-04-30T06:43:49.879ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "01ae9bba-0126-4c61-94a8-c57319e334b3", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
39082025-04-30T06:43:49.879ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
39092025-04-30T06:43:49.881ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b60c2ac2-e486-4100-ae93-a04204456ca8", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
39102025-04-30T06:43:49.881ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
39112025-04-30T06:43:49.886ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bfe2c8e7-6cbe-4e32-a1ef-b990347f3466", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
39122025-04-30T06:43:49.886ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
39132025-04-30T06:43:49.890ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
39142025-04-30T06:43:49.898ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5c6fcf8b-af60-4f51-ac8f-917009d1da06", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
39152025-04-30T06:43:49.898ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
39162025-04-30T06:43:49.900ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f1e3d878-7134-4cf2-a120-698be7b0b99c", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
39172025-04-30T06:43:49.900ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
39182025-04-30T06:43:49.902ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "276ee307-6766-4aee-9ecc-169268abb3b7", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
39192025-04-30T06:43:49.902ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
39202025-04-30T06:43:49.910ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d490a067-dbd5-465c-a822-a83765afe123", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
39212025-04-30T06:43:49.910ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
39222025-04-30T06:43:49.911ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f9804f39-781b-423a-a95c-2f85ad599e81", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
39232025-04-30T06:43:49.912ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
39242025-04-30T06:43:49.914ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a1e458a9-c573-4dfb-b8c8-88941d189083", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
39252025-04-30T06:43:49.914ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
39262025-04-30T06:43:49.918ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "958b195b-8d1b-4d33-adb5-fdec6a944b3e", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
39272025-04-30T06:43:49.918ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
39282025-04-30T06:43:49.921ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3e2d0554-f129-425e-a1fc-cb41fa85475d", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
39292025-04-30T06:43:49.921ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
39302025-04-30T06:43:49.924ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6fe8e2a5-f8a5-4c94-88d2-ec95fd26de30", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
39312025-04-30T06:43:49.924ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
39322025-04-30T06:43:49.928ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "45cd50da-1c26-452b-bd3f-668191d99c30", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
39332025-04-30T06:43:49.928ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
39342025-04-30T06:43:49.933ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8908bb27-296c-4de1-9f7d-50f272e8cb7f", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
39352025-04-30T06:43:49.935ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
39362025-04-30T06:43:49.938ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "93cbe08e-3b0d-472e-941e-430764404140", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
39372025-04-30T06:43:49.939ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
39382025-04-30T06:43:49.940ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7d3d9cb7-8b97-4f45-ae5b-454c7359b8d8", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
39392025-04-30T06:43:49.940ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
39402025-04-30T06:43:49.941ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
39412025-04-30T06:43:49.941ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
39422025-04-30T06:43:49.941ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
39432025-04-30T06:43:49.941ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
39442025-04-30T06:43:49.942ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
39452025-04-30T06:43:49.942ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3a9e9418-2957-42c1-82ae-551f5ee6b75e", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
39462025-04-30T06:43:49.942ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
39472025-04-30T06:43:49.944ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "dcd26ff6-9e2c-444c-a441-8cb229574a04", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
39482025-04-30T06:43:49.944ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
39492025-04-30T06:43:49.945ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39502025-04-30T06:43:49.953ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
39512025-04-30T06:43:49.953ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
39522025-04-30T06:43:49.953ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
39532025-04-30T06:43:49.954ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
39542025-04-30T06:43:49.956ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c4e39ece-6ea9-468c-94b7-37eee68f0134", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
39552025-04-30T06:43:49.956ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
39562025-04-30T06:43:49.958ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
39572025-04-30T06:43:49.958ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
39582025-04-30T06:43:49.959ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
39592025-04-30T06:43:49.972ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5524dfe7-85aa-4415-87af-824afb4f37a2", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
39602025-04-30T06:43:49.972ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
39612025-04-30T06:43:49.974ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "81476b83-7391-428d-bfa7-cec85bc73d8a", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
39622025-04-30T06:43:49.974ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
39632025-04-30T06:43:49.977ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "36fa5ab9-eeb8-4dbe-b84c-ccb32c3d9587", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
39642025-04-30T06:43:49.977ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
39652025-04-30T06:43:49.980ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "47d77acc-49f1-496c-aaaa-83784d2e3013", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
39662025-04-30T06:43:49.980ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
39672025-04-30T06:43:49.982ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2e84b400-e1e5-49ec-b8bf-87f9d5b5321a", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
39682025-04-30T06:43:49.983ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
39692025-04-30T06:43:49.989ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c034cb77-a4bd-4d74-ac1d-e48a7c4f727e", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
39702025-04-30T06:43:49.989ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
39712025-04-30T06:43:49.997ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3d8c5bd1-b3d4-41f8-a9a1-9d51ff95065e", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
39722025-04-30T06:43:50.009ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
39732025-04-30T06:43:50.010ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "50f9c1e3-ae7e-4278-bb71-8d999f0caefc", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
39742025-04-30T06:43:50.011ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
39752025-04-30T06:43:50.012ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0300bffe-a02e-4017-a2ab-bb11a5d79c7f", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
39762025-04-30T06:43:50.012ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
39772025-04-30T06:43:50.016ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
39782025-04-30T06:43:50.016ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c16cb7bb-6c27-4697-bc8a-f6ba252b461b", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
39792025-04-30T06:43:50.016ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
39802025-04-30T06:43:50.028ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "366e9e42-5c41-4f6c-80c7-6ed457fe921b", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
39812025-04-30T06:43:50.028ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
39822025-04-30T06:43:50.029ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1aa1fffc-f971-4581-8ce0-86de2c3a39ea", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
39832025-04-30T06:43:50.030ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
39842025-04-30T06:43:50.033ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4edc1201-29e2-4207-8a63-813f07ae1847", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
39852025-04-30T06:43:50.033ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
39862025-04-30T06:43:50.035ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c1414994-6b33-4363-b8fe-e8c2acca1d1d", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
39872025-04-30T06:43:50.035ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
39882025-04-30T06:43:50.037ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "af56fdcb-04d2-4c2d-9ca6-912a3a58b676", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
39892025-04-30T06:43:50.037ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
39902025-04-30T06:43:50.039ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ecc9e7aa-fa5b-4f19-b170-4cf997f67791", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
39912025-04-30T06:43:50.039ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
39922025-04-30T06:43:50.041ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f6310645-ad7f-483f-8825-9ecb754c2e9b", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
39932025-04-30T06:43:50.042ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
39942025-04-30T06:43:50.043ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b5bb93fd-d947-4c37-aca2-c82a8e89b885", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
39952025-04-30T06:43:50.044ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
39962025-04-30T06:43:50.044ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a4a6c879-d71f-4325-921e-b32a3022b3d4", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
39972025-04-30T06:43:50.045ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
39982025-04-30T06:43:50.054ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3832759e-ca20-4a1d-b41d-ef6f317a63ad", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
39992025-04-30T06:43:50.054ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
40002025-04-30T06:43:50.055ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ca80886a-6ea7-4fb6-be83-0f0e74e5a7c1", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
40012025-04-30T06:43:50.055ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
40022025-04-30T06:43:50.056ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "10c15431-0cb7-4cf2-a52e-5330bff74aed", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
40032025-04-30T06:43:50.056ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
40042025-04-30T06:43:50.056ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
40052025-04-30T06:43:50.056ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
40062025-04-30T06:43:50.056ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
40072025-04-30T06:43:50.057ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
40082025-04-30T06:43:50.062ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40092025-04-30T06:43:50.066ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
40102025-04-30T06:43:50.066ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
40112025-04-30T06:43:50.066ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
40122025-04-30T06:43:50.066ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
40132025-04-30T06:43:50.066ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
40142025-04-30T06:43:50.066ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
40152025-04-30T06:43:50.067ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
40162025-04-30T06:43:50.067ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
40172025-04-30T06:43:50.067ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
40182025-04-30T06:43:50.067ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
40192025-04-30T06:43:50.067ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
40202025-04-30T06:43:50.070ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40212025-04-30T06:43:50.073ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
40222025-04-30T06:43:50.073ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
40232025-04-30T06:43:50.073ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
40242025-04-30T06:43:50.073ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
40252025-04-30T06:43:50.075ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40262025-04-30T06:43:50.081ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
40272025-04-30T06:43:50.088ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
40282025-04-30T06:43:50.088ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
40292025-04-30T06:43:50.089ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
40302025-04-30T06:43:50.092ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
40312025-04-30T06:43:50.092ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
40322025-04-30T06:43:50.093ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
40332025-04-30T06:43:50.161ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
40342025-04-30T06:43:50.162ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory
40352025-04-30T06:43:50.162ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:36547 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
40362025-04-30T06:43:50.163ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:36547 req_id = d9ef1d76-72f2-4a73-8590-76b78d7b778c sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
40372025-04-30T06:43:50.164ZINFOtest_unauthorized (dropshot): request completed latency_us = 990 local_addr = [::1]:57786 method = GET remote_addr = [::1]:36547 req_id = d9ef1d76-72f2-4a73-8590-76b78d7b778c response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
40382025-04-30T06:43:50.164ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = http://[::1]:57786 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:57786/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "d9ef1d76-72f2-4a73-8590-76b78d7b778c", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
40392025-04-30T06:43:50.167ZDEBGtest_unauthorized (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
40402025-04-30T06:43:50.167ZDEBGtest_unauthorized (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
40412025-04-30T06:43:50.167ZDEBGtest_unauthorized (BackgroundTasks): finished collection background_task = inventory_collection
40422025-04-30T06:43:50.169ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
40432025-04-30T06:43:50.169ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
40442025-04-30T06:43:50.169ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
40452025-04-30T06:43:50.169ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
40462025-04-30T06:43:50.170ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
40472025-04-30T06:43:50.173ZTRACtest_unauthorized (results-sink): batch interval expired, but no samples to insert collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
40482025-04-30T06:43:50.173ZTRACtest_unauthorized (results-sink-cluster): batch interval expired, but no samples to insert collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
40492025-04-30T06:43:50.173ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40502025-04-30T06:43:50.179ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
40512025-04-30T06:43:50.179ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
40522025-04-30T06:43:50.180ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
40532025-04-30T06:43:50.180ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
40542025-04-30T06:43:50.203ZDEBGtest_unauthorized (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
40552025-04-30T06:43:50.203ZDEBGtest_unauthorized (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
40562025-04-30T06:43:50.209ZTRACtest_unauthorized (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
40572025-04-30T06:43:50.210ZDEBGtest_unauthorized (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
40582025-04-30T06:43:50.210ZTRACtest_unauthorized (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
40592025-04-30T06:43:50.210ZTRACtest_unauthorized (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
40602025-04-30T06:43:50.221ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
40612025-04-30T06:43:50.221ZWARNtest_unauthorized (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
40622025-04-30T06:43:50.221ZWARNtest_unauthorized (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
40632025-04-30T06:43:50.221ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 581.553247ms iteration = 7 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
40642025-04-30T06:43:50.221ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 8 reason = Signaled
40652025-04-30T06:43:50.221ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
40662025-04-30T06:43:50.221ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
40672025-04-30T06:43:50.221ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
40682025-04-30T06:43:50.221ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
40692025-04-30T06:43:50.222ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
40702025-04-30T06:43:50.227ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40712025-04-30T06:43:50.231ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
40722025-04-30T06:43:50.231ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
40732025-04-30T06:43:50.231ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
40742025-04-30T06:43:50.231ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
40752025-04-30T06:43:50.234ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1a8b2318-abf8-406d-a389-112ab91a0de8, 0)]
40762025-04-30T06:43:50.234ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
40772025-04-30T06:43:50.235ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 1a 93 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:55932 req_id = 8b378a62-0eee-4112-a67b-1c9fc576ca17
40782025-04-30T06:43:50.235ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 6803,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:55932 req_id = 8b378a62-0eee-4112-a67b-1c9fc576ca17
40792025-04-30T06:43:50.235ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
40802025-04-30T06:43:50.236ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
40812025-04-30T06:43:50.236ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:55932 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 8b378a62-0eee-4112-a67b-1c9fc576ca17
40822025-04-30T06:43:50.236ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4295, tv_nsec: 35512889 } })
40832025-04-30T06:43:50.236ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
40842025-04-30T06:43:50.236ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: cb 05 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:51059 req_id = 9d4b1759-a8ea-49b2-87f2-543f6fbc39bc
40852025-04-30T06:43:50.237ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 51973,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51059 req_id = 9d4b1759-a8ea-49b2-87f2-543f6fbc39bc
40862025-04-30T06:43:50.237ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
40872025-04-30T06:43:50.237ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:51059 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 9d4b1759-a8ea-49b2-87f2-543f6fbc39bc
40882025-04-30T06:43:50.277ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
40892025-04-30T06:43:50.277ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
40902025-04-30T06:43:50.278ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
40912025-04-30T06:43:50.278ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
40922025-04-30T06:43:50.278ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
40932025-04-30T06:43:50.279ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
40942025-04-30T06:43:50.280ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 6e 30 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c n0..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:55536 req_id = 261f0b43-5115-4555-8b58-3d0aa0cb2087
40952025-04-30T06:43:50.280ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 28208,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:55536 req_id = 261f0b43-5115-4555-8b58-3d0aa0cb2087
40962025-04-30T06:43:50.280ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
40972025-04-30T06:43:50.280ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 28208, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:55536 req_id = 261f0b43-5115-4555-8b58-3d0aa0cb2087
40982025-04-30T06:43:50.281ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
40992025-04-30T06:43:50.281ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
41002025-04-30T06:43:50.281ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
41012025-04-30T06:43:50.282ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41022025-04-30T06:43:50.295ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
41032025-04-30T06:43:50.295ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
41042025-04-30T06:43:50.295ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
41052025-04-30T06:43:50.295ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
41062025-04-30T06:43:50.297ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "5fd7cf43-3c48-49e2-9c9e-94247d2abcdd", "content-length": "835", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
41072025-04-30T06:43:50.297ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
41082025-04-30T06:43:50.298ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
41092025-04-30T06:43:50.298ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
41102025-04-30T06:43:50.298ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
41112025-04-30T06:43:50.299ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3bf37c4d-b9e7-49e6-8873-68534a2aeca8", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
41122025-04-30T06:43:50.300ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
41132025-04-30T06:43:50.302ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b8a46c37-3945-40b5-88f9-228ef733a737", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
41142025-04-30T06:43:50.302ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
41152025-04-30T06:43:50.305ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "577b0f78-97e4-430c-8b91-04248f28bf93", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
41162025-04-30T06:43:50.305ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
41172025-04-30T06:43:50.307ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "11d1d277-76c7-4d3e-9b7b-df72fdd07de9", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
41182025-04-30T06:43:50.307ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
41192025-04-30T06:43:50.309ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c92552a5-c440-4cf1-87b3-41f178134c68", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
41202025-04-30T06:43:50.309ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
41212025-04-30T06:43:50.311ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "40eab4b9-218d-40cb-b44d-75babee4b071", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
41222025-04-30T06:43:50.311ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
41232025-04-30T06:43:50.316ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e5803386-7987-425e-99e7-96f6ea84da52", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
41242025-04-30T06:43:50.316ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
41252025-04-30T06:43:50.316ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "00b0ca80-e21a-4e2b-ad72-bf0db5d978de", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
41262025-04-30T06:43:50.316ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
41272025-04-30T06:43:50.321ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "dc7170eb-5308-4ddc-aba2-68a62d5c36c3", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
41282025-04-30T06:43:50.321ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
41292025-04-30T06:43:50.322ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "73b08e32-f451-4140-a5cc-31fd6663fe0a", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
41302025-04-30T06:43:50.322ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
41312025-04-30T06:43:50.323ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fe20c7c4-008c-45e4-80b6-411d431ab5dc", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
41322025-04-30T06:43:50.323ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
41332025-04-30T06:43:50.325ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "197fbad4-41f7-480d-a0f6-506aa4fe7f43", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
41342025-04-30T06:43:50.325ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
41352025-04-30T06:43:50.347ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2ed4eefe-0729-49b0-a1e7-c98b6b1eb223", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
41362025-04-30T06:43:50.354ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
41372025-04-30T06:43:50.355ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
41382025-04-30T06:43:50.358ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bd298cfc-b504-4143-9104-9637c1855aad", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
41392025-04-30T06:43:50.358ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
41402025-04-30T06:43:50.360ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "edc22b2a-e292-4547-8d53-265cfadee334", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
41412025-04-30T06:43:50.360ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
41422025-04-30T06:43:50.362ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b9a1aea1-093b-44e7-a257-26fed2684434", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
41432025-04-30T06:43:50.362ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
41442025-04-30T06:43:50.364ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cf4a32ef-bea0-444f-b7c8-0f732d3e6037", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
41452025-04-30T06:43:50.364ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
41462025-04-30T06:43:50.366ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f23e47ad-d2cc-410a-8b86-9b3f4a579ef5", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
41472025-04-30T06:43:50.366ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
41482025-04-30T06:43:50.367ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "62d3e0a6-ad13-40a1-a454-4c024336193e", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
41492025-04-30T06:43:50.367ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
41502025-04-30T06:43:50.376ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "857518b5-b321-4302-8abf-6b801f4e08e5", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
41512025-04-30T06:43:50.376ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
41522025-04-30T06:43:50.377ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a07c854b-8868-4557-86fe-d0fca9dc77a6", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
41532025-04-30T06:43:50.377ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
41542025-04-30T06:43:50.378ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ab3fe5ea-d3aa-420f-852d-0bb7556ae289", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
41552025-04-30T06:43:50.378ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
41562025-04-30T06:43:50.379ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "740a27be-d5af-4671-96ec-cc600b8b95a9", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
41572025-04-30T06:43:50.379ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
41582025-04-30T06:43:50.381ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c249c46d-a7f4-4156-aa54-ead609cc1823", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
41592025-04-30T06:43:50.381ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
41602025-04-30T06:43:50.383ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9aeb15ec-116f-47bf-9871-548a2b964090", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
41612025-04-30T06:43:50.383ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
41622025-04-30T06:43:50.389ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9b9bb8a0-189f-4c46-9d2a-0072dd2e028a", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
41632025-04-30T06:43:50.392ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
41642025-04-30T06:43:50.395ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9ad97b7f-2c49-43f4-bb7b-264baea610b8", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
41652025-04-30T06:43:50.395ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
41662025-04-30T06:43:50.397ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e2037d20-873e-4ac1-9397-41bc984448d2", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
41672025-04-30T06:43:50.397ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
41682025-04-30T06:43:50.399ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0b60b1a8-286a-4c1a-841f-2f51fbc994b1", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
41692025-04-30T06:43:50.399ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
41702025-04-30T06:43:50.400ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e48e58f9-3779-4ec2-8f48-c30a26eb1cfa", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
41712025-04-30T06:43:50.400ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
41722025-04-30T06:43:50.401ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1d0a3a08-2ff3-488d-883b-74371e292dea", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
41732025-04-30T06:43:50.402ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
41742025-04-30T06:43:50.403ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "136ff0c5-cb61-4fcd-a394-8678c68efe15", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
41752025-04-30T06:43:50.403ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
41762025-04-30T06:43:50.404ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "064dd446-6d95-4c34-a48b-2bb5646af6c1", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
41772025-04-30T06:43:50.404ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
41782025-04-30T06:43:50.405ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "aa8ea193-1899-408a-aabc-601c86ea8b1a", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
41792025-04-30T06:43:50.405ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
41802025-04-30T06:43:50.406ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
41812025-04-30T06:43:50.406ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
41822025-04-30T06:43:50.406ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
41832025-04-30T06:43:50.410ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
41842025-04-30T06:43:50.411ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
41852025-04-30T06:43:50.413ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41862025-04-30T06:43:50.418ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
41872025-04-30T06:43:50.418ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
41882025-04-30T06:43:50.418ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
41892025-04-30T06:43:50.419ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
41902025-04-30T06:43:50.419ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "412dcc1c-bd56-442c-b01f-03f5050306ad", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
41912025-04-30T06:43:50.419ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
41922025-04-30T06:43:50.422ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
41932025-04-30T06:43:50.422ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
41942025-04-30T06:43:50.422ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
41952025-04-30T06:43:50.423ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a6794cab-f38f-462e-82bc-929626fd228c", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
41962025-04-30T06:43:50.423ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
41972025-04-30T06:43:50.426ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5c3c8295-f858-4f89-ae1d-6de64a0387c2", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
41982025-04-30T06:43:50.427ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
41992025-04-30T06:43:50.429ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e7929209-a9a1-4e98-bfcd-24c565935b8c", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
42002025-04-30T06:43:50.430ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
42012025-04-30T06:43:50.432ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "095dcc4c-148b-4cfd-a12f-9197c83ed1fe", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
42022025-04-30T06:43:50.432ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
42032025-04-30T06:43:50.434ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f11dadfb-69b1-4288-bc47-1412b4d5eb35", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
42042025-04-30T06:43:50.434ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
42052025-04-30T06:43:50.436ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "80b1c0f8-5b10-4557-89f9-c94e3e993ec2", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
42062025-04-30T06:43:50.436ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
42072025-04-30T06:43:50.438ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "82911512-4b4c-431b-bd28-31e1eada4277", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
42082025-04-30T06:43:50.438ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
42092025-04-30T06:43:50.439ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6270ce3c-28b6-42ec-8970-2b09474b5a94", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
42102025-04-30T06:43:50.439ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
42112025-04-30T06:43:50.441ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "eeb92556-5f92-4121-8b5f-e91c9ef6046b", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
42122025-04-30T06:43:50.441ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
42132025-04-30T06:43:50.441ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
42142025-04-30T06:43:50.441ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
42152025-04-30T06:43:50.441ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
42162025-04-30T06:43:50.443ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
42172025-04-30T06:43:50.454ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42182025-04-30T06:43:50.459ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
42192025-04-30T06:43:50.460ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
42202025-04-30T06:43:50.461ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
42212025-04-30T06:43:50.461ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
42222025-04-30T06:43:50.461ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
42232025-04-30T06:43:50.462ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
42242025-04-30T06:43:50.464ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42252025-04-30T06:43:50.471ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
42262025-04-30T06:43:50.471ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
42272025-04-30T06:43:50.471ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
42282025-04-30T06:43:50.475ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
42292025-04-30T06:43:50.539ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
42302025-04-30T06:43:50.539ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory
42312025-04-30T06:43:50.540ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
42322025-04-30T06:43:50.540ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:56951 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
42332025-04-30T06:43:50.573ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:56951 req_id = b231e57c-4157-457a-9068-e292a73314a7 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
42342025-04-30T06:43:50.573ZINFOtest_unauthorized (dropshot): request completed latency_us = 772 local_addr = [::1]:57786 method = GET remote_addr = [::1]:56951 req_id = b231e57c-4157-457a-9068-e292a73314a7 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
42352025-04-30T06:43:50.575ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = http://[::1]:57786 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:57786/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "b231e57c-4157-457a-9068-e292a73314a7", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
42362025-04-30T06:43:50.579ZDEBGtest_unauthorized (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
42372025-04-30T06:43:50.579ZDEBGtest_unauthorized (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
42382025-04-30T06:43:50.579ZDEBGtest_unauthorized (BackgroundTasks): finished collection background_task = inventory_collection
42392025-04-30T06:43:50.582ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
42402025-04-30T06:43:50.583ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
42412025-04-30T06:43:50.583ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
42422025-04-30T06:43:50.583ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
42432025-04-30T06:43:50.584ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
42442025-04-30T06:43:50.619ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42452025-04-30T06:43:50.623ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
42462025-04-30T06:43:50.623ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
42472025-04-30T06:43:50.623ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
42482025-04-30T06:43:50.624ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
42492025-04-30T06:43:50.625ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
42502025-04-30T06:43:50.625ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
42512025-04-30T06:43:50.625ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
42522025-04-30T06:43:50.625ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
42532025-04-30T06:43:50.625ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
42542025-04-30T06:43:50.642ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42552025-04-30T06:43:50.647ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
42562025-04-30T06:43:50.647ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
42572025-04-30T06:43:50.648ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
42582025-04-30T06:43:50.648ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
42592025-04-30T06:43:50.652ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
42602025-04-30T06:43:50.652ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
42612025-04-30T06:43:50.652ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
42622025-04-30T06:43:50.761ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
42632025-04-30T06:43:50.786ZWARNtest_unauthorized (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
42642025-04-30T06:43:50.793ZWARNtest_unauthorized (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
42652025-04-30T06:43:50.793ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 571.451109ms iteration = 8 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
42662025-04-30T06:43:50.793ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 9 reason = Signaled
42672025-04-30T06:43:50.793ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
42682025-04-30T06:43:50.793ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
42692025-04-30T06:43:50.793ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
42702025-04-30T06:43:50.793ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
42712025-04-30T06:43:50.794ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
42722025-04-30T06:43:50.797ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42732025-04-30T06:43:50.800ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
42742025-04-30T06:43:50.800ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
42752025-04-30T06:43:50.800ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
42762025-04-30T06:43:50.800ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
42772025-04-30T06:43:50.822ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
42782025-04-30T06:43:50.822ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
42792025-04-30T06:43:50.822ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
42802025-04-30T06:43:50.822ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
42812025-04-30T06:43:50.822ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
42822025-04-30T06:43:50.823ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1a8b2318-abf8-406d-a389-112ab91a0de8, 0)]
42832025-04-30T06:43:50.823ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
42842025-04-30T06:43:50.827ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: c1 0b 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:61498 req_id = cb83e00b-1907-44fd-ae65-952353a8857a
42852025-04-30T06:43:50.827ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 49419,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:61498 req_id = cb83e00b-1907-44fd-ae65-952353a8857a
42862025-04-30T06:43:50.827ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
42872025-04-30T06:43:50.827ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
42882025-04-30T06:43:50.827ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:61498 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = cb83e00b-1907-44fd-ae65-952353a8857a
42892025-04-30T06:43:50.828ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42902025-04-30T06:43:50.837ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
42912025-04-30T06:43:50.842ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
42922025-04-30T06:43:50.843ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
42932025-04-30T06:43:50.843ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
42942025-04-30T06:43:50.843ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4295, tv_nsec: 642834357 } })
42952025-04-30T06:43:50.844ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
42962025-04-30T06:43:50.855ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: cc 26 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .&..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:59841 req_id = 981b88a3-74ed-4fc6-b644-01576f4745cc
42972025-04-30T06:43:50.856ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 52262,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:59841 req_id = 981b88a3-74ed-4fc6-b644-01576f4745cc
42982025-04-30T06:43:50.856ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
42992025-04-30T06:43:50.857ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:59841 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 981b88a3-74ed-4fc6-b644-01576f4745cc
43002025-04-30T06:43:50.946ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
43012025-04-30T06:43:50.950ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
43022025-04-30T06:43:50.951ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
43032025-04-30T06:43:50.952ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
43042025-04-30T06:43:50.953ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: d6 81 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:51635 req_id = 229b15aa-52c8-400a-9354-6695de26e9a1
43052025-04-30T06:43:50.953ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 54913,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51635 req_id = 229b15aa-52c8-400a-9354-6695de26e9a1
43062025-04-30T06:43:50.953ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
43072025-04-30T06:43:50.953ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 54913, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:51635 req_id = 229b15aa-52c8-400a-9354-6695de26e9a1
43082025-04-30T06:43:50.953ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
43092025-04-30T06:43:50.953ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
43102025-04-30T06:43:50.953ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
43112025-04-30T06:43:50.955ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "848ed8d6-d6c2-42f1-9fb9-879073629391", "content-length": "835", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
43122025-04-30T06:43:50.955ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
43132025-04-30T06:43:50.957ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fec1826c-342d-49e3-bade-16181540fb12", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
43142025-04-30T06:43:50.958ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
43152025-04-30T06:43:50.962ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e2171770-0cb6-4ef2-88a9-74f60017d964", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
43162025-04-30T06:43:50.962ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
43172025-04-30T06:43:50.967ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3fe5d593-f2a7-40b6-8af7-176ae22ce47e", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
43182025-04-30T06:43:50.968ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
43192025-04-30T06:43:50.974ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ae3c82fa-ef0c-42c6-804b-8a47d08d56a3", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
43202025-04-30T06:43:50.974ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
43212025-04-30T06:43:50.990ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2315793c-2cf1-4ffe-9d0d-1b564d94aed2", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
43222025-04-30T06:43:50.992ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
43232025-04-30T06:43:51.034ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
43242025-04-30T06:43:51.043ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "544e0a8b-00d2-41d0-805d-e3e9130cb9fa", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
43252025-04-30T06:43:51.043ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
43262025-04-30T06:43:51.046ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "573d8bf4-6fdf-4897-87ff-55e8efbfb015", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
43272025-04-30T06:43:51.047ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
43282025-04-30T06:43:51.048ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "49e33ffa-a911-4388-ae9b-240212a89bc4", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
43292025-04-30T06:43:51.048ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
43302025-04-30T06:43:51.049ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3724edaa-bc7f-41e9-9a38-5dfa152401f3", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
43312025-04-30T06:43:51.049ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
43322025-04-30T06:43:51.050ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "03a68701-650f-409d-be07-05fbd68bd1c8", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
43332025-04-30T06:43:51.050ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
43342025-04-30T06:43:51.051ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "190df6e3-2638-4f13-bea3-ec09d8e20f63", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
43352025-04-30T06:43:51.051ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
43362025-04-30T06:43:51.053ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "354be915-34ad-4a8c-bd65-0eeaf01e14e6", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
43372025-04-30T06:43:51.053ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
43382025-04-30T06:43:51.055ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "85e8f25f-fcec-438c-8912-dfbad04518a6", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
43392025-04-30T06:43:51.055ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
43402025-04-30T06:43:51.057ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fbc795dd-4401-4eed-8b40-ddd41283bc17", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
43412025-04-30T06:43:51.057ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
43422025-04-30T06:43:51.059ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1470f7f4-4e72-4c36-8f5e-207094dce872", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
43432025-04-30T06:43:51.059ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
43442025-04-30T06:43:51.062ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "47d1b205-7664-4410-82ef-2f937c08eae1", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
43452025-04-30T06:43:51.062ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
43462025-04-30T06:43:51.065ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3c0c064c-4027-4ae0-a0c1-43bc47b32bc3", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
43472025-04-30T06:43:51.065ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
43482025-04-30T06:43:51.066ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d99e7bb9-da4b-4d1a-9668-85e7c270d441", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
43492025-04-30T06:43:51.066ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
43502025-04-30T06:43:51.067ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5947c344-241f-4caa-8c39-89a51a02e560", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
43512025-04-30T06:43:51.067ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
43522025-04-30T06:43:51.071ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d5df2bf3-6aed-448d-9286-5cddc072dc4a", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
43532025-04-30T06:43:51.071ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
43542025-04-30T06:43:51.072ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "65fce991-8a53-4466-89a6-b844f43923c1", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
43552025-04-30T06:43:51.072ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
43562025-04-30T06:43:51.075ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c8bd0d3e-1940-4e82-89ec-fb962a01f69c", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
43572025-04-30T06:43:51.075ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
43582025-04-30T06:43:51.078ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "266bcecf-4b25-43b7-955a-4c92bb0d34a1", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
43592025-04-30T06:43:51.078ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
43602025-04-30T06:43:51.078ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ea357a52-996f-47fe-8436-117c0ddc502c", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
43612025-04-30T06:43:51.079ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
43622025-04-30T06:43:51.081ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "44bba52b-5740-45d5-a417-fb1cd74494ee", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
43632025-04-30T06:43:51.081ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
43642025-04-30T06:43:51.084ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f13731cb-8676-4d44-88b2-669659e9d676", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
43652025-04-30T06:43:51.084ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
43662025-04-30T06:43:51.085ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
43672025-04-30T06:43:51.085ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
43682025-04-30T06:43:51.085ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
43692025-04-30T06:43:51.085ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
43702025-04-30T06:43:51.086ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
43712025-04-30T06:43:51.087ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "45d28fe1-b73d-4b69-8c34-62329b01e70a", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
43722025-04-30T06:43:51.087ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
43732025-04-30T06:43:51.088ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43742025-04-30T06:43:51.093ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
43752025-04-30T06:43:51.093ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
43762025-04-30T06:43:51.093ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
43772025-04-30T06:43:51.093ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
43782025-04-30T06:43:51.095ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "103abfc7-e13d-409d-9486-7f90a8a71394", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
43792025-04-30T06:43:51.095ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
43802025-04-30T06:43:51.096ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
43812025-04-30T06:43:51.096ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
43822025-04-30T06:43:51.096ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
43832025-04-30T06:43:51.098ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "47b3d724-b9a9-4814-b88c-45fdc8d3ab2d", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
43842025-04-30T06:43:51.098ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
43852025-04-30T06:43:51.099ZINFOtest_unauthorized (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
43862025-04-30T06:43:51.099ZDEBGtest_unauthorized (oximeter): client request body = None method = GET uri = http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
43872025-04-30T06:43:51.100ZTRACtest_unauthorized (dropshot_internal): incoming request local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = d9cfd2f2-59d6-4773-8649-9760ddc3b147 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
43882025-04-30T06:43:51.100ZTRACtest_unauthorized (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = d9cfd2f2-59d6-4773-8649-9760ddc3b147 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
43892025-04-30T06:43:51.100ZDEBGtest_unauthorized (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = d9cfd2f2-59d6-4773-8649-9760ddc3b147 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
43902025-04-30T06:43:51.100ZDEBGtest_unauthorized (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = d9cfd2f2-59d6-4773-8649-9760ddc3b147 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
43912025-04-30T06:43:51.101ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7c7e25ea-56cd-46ce-a0ad-1f8af6e97b27", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
43922025-04-30T06:43:51.102ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
43932025-04-30T06:43:51.104ZINFOtest_unauthorized (dropshot_internal): request completed latency_us = 4809 local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = d9cfd2f2-59d6-4773-8649-9760ddc3b147 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
43942025-04-30T06:43:51.105ZDEBGtest_unauthorized (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "d9cfd2f2-59d6-4773-8649-9760ddc3b147", "content-length": "405", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
43952025-04-30T06:43:51.105ZDEBGtest_unauthorized (oximeter): client request body = None method = GET uri = http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
43962025-04-30T06:43:51.106ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "28e101b0-6617-4077-ad6a-dc624aeaf50b", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
43972025-04-30T06:43:51.106ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
43982025-04-30T06:43:51.106ZTRACtest_unauthorized (dropshot_internal): incoming request local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 05939ec4-30f1-4ff2-ab33-5f4deefda560 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
43992025-04-30T06:43:51.107ZTRACtest_unauthorized (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 05939ec4-30f1-4ff2-ab33-5f4deefda560 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
44002025-04-30T06:43:51.107ZDEBGtest_unauthorized (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 05939ec4-30f1-4ff2-ab33-5f4deefda560 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
44012025-04-30T06:43:51.111ZDEBGtest_unauthorized (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 05939ec4-30f1-4ff2-ab33-5f4deefda560 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
44022025-04-30T06:43:51.112ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fe0b522e-952f-44e1-bb19-6e6836393009", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
44032025-04-30T06:43:51.112ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
44042025-04-30T06:43:51.116ZINFOtest_unauthorized (dropshot_internal): request completed latency_us = 9629 local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 05939ec4-30f1-4ff2-ab33-5f4deefda560 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
44052025-04-30T06:43:51.116ZDEBGtest_unauthorized (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "05939ec4-30f1-4ff2-ab33-5f4deefda560", "content-length": "29", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
44062025-04-30T06:43:51.116ZTRACtest_unauthorized (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 8d752adb-08b9-405f-a11b-66019bf447b5
44072025-04-30T06:43:51.116ZTRACtest_unauthorized (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
44082025-04-30T06:43:51.116ZINFOtest_unauthorized (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
44092025-04-30T06:43:51.116ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "503854f7-b9cf-42bf-8fd1-015b1e1626a4", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
44102025-04-30T06:43:51.117ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
44112025-04-30T06:43:51.119ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b310272b-8a03-47ad-8011-f7c4caa470a8", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
44122025-04-30T06:43:51.119ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
44132025-04-30T06:43:51.121ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9decbfe8-9f44-43a3-8d50-6adb1d120771", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
44142025-04-30T06:43:51.121ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
44152025-04-30T06:43:51.124ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fbda91f2-f23f-4a33-aadd-fe0eb9f3201e", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
44162025-04-30T06:43:51.124ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
44172025-04-30T06:43:51.125ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e41178a6-c106-4126-8a09-fa3047a84868", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
44182025-04-30T06:43:51.125ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
44192025-04-30T06:43:51.127ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "85cde6c2-80c2-48fb-a98a-43c041174528", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
44202025-04-30T06:43:51.127ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
44212025-04-30T06:43:51.130ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "32dc156e-9fcc-4fbb-8d34-842381037344", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
44222025-04-30T06:43:51.130ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
44232025-04-30T06:43:51.132ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0812500d-d61b-4125-8ad6-1dcfbc5d31a3", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
44242025-04-30T06:43:51.132ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
44252025-04-30T06:43:51.135ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0e57293f-6697-475f-b94a-6af0dea4c29a", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
44262025-04-30T06:43:51.136ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
44272025-04-30T06:43:51.138ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d1e38db4-2126-411b-a853-68681a4b94bc", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
44282025-04-30T06:43:51.138ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
44292025-04-30T06:43:51.142ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1877cd5e-7bd7-40a2-a558-a69906c034c3", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
44302025-04-30T06:43:51.142ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
44312025-04-30T06:43:51.142ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
44322025-04-30T06:43:51.143ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "963edf13-afbe-455b-91bc-450307f53b25", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
44332025-04-30T06:43:51.143ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
44342025-04-30T06:43:51.144ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
44352025-04-30T06:43:51.144ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
44362025-04-30T06:43:51.144ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
44372025-04-30T06:43:51.144ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
44382025-04-30T06:43:51.150ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44392025-04-30T06:43:51.166ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
44402025-04-30T06:43:51.167ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
44412025-04-30T06:43:51.167ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
44422025-04-30T06:43:51.167ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
44432025-04-30T06:43:51.167ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
44442025-04-30T06:43:51.167ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
44452025-04-30T06:43:51.169ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44462025-04-30T06:43:51.176ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
44472025-04-30T06:43:51.176ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
44482025-04-30T06:43:51.176ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
44492025-04-30T06:43:51.176ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
44502025-04-30T06:43:51.178ZDEBGtest_unauthorized (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
44512025-04-30T06:43:51.178ZDEBGtest_unauthorized (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
44522025-04-30T06:43:51.178ZDEBGtest_unauthorized (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a
44532025-04-30T06:43:51.179ZDEBGtest_unauthorized (clickhouse-client): retrieving timeseries schema from database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a
44542025-04-30T06:43:51.179ZTRACtest_unauthorized (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a sql = SELECT * FROM oximeter.timeseries_schema FORMAT Native;
44552025-04-30T06:43:51.181ZTRACtest_unauthorized (clickhouse-client): no new timeseries schema in database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a
44562025-04-30T06:43:51.182ZDEBGtest_unauthorized (clickhouse-client): new timeseries schema collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a schema = TimeseriesSchema { timeseries_name: TimeseriesName("integration_target:integration_metric"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "metric_name", field_type: String, source: Metric, description: "" }, FieldSchema { name: "target_name", field_type: String, source: Target, description: "" }}, datum_type: I64, version: 1, authz_scope: Fleet, units: Count, created: 2025-04-30T06:43:51.179000679Z } timeseries_name = integration_target:integration_metric
44572025-04-30T06:43:51.182ZDEBGtest_unauthorized (clickhouse-client): inserting 1 new timeseries schema collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a
44582025-04-30T06:43:51.182ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 6 n_rows = 1 sql = INSERT INTO oximeter.timeseries_schema FORMAT Native
44592025-04-30T06:43:51.248ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
44602025-04-30T06:43:51.248ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory
44612025-04-30T06:43:51.248ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
44622025-04-30T06:43:51.248ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
44632025-04-30T06:43:51.249ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
44642025-04-30T06:43:51.249ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
44652025-04-30T06:43:51.253ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
44662025-04-30T06:43:51.253ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:62454 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
44672025-04-30T06:43:51.253ZDEBGtest_unauthorized (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
44682025-04-30T06:43:51.253ZDEBGtest_unauthorized (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
44692025-04-30T06:43:51.258ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44702025-04-30T06:43:51.264ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
44712025-04-30T06:43:51.264ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
44722025-04-30T06:43:51.264ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
44732025-04-30T06:43:51.268ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
44742025-04-30T06:43:51.269ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
44752025-04-30T06:43:51.271ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:62454 req_id = 1c440ef8-203c-4857-9b6a-f86a4a7fee82 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
44762025-04-30T06:43:51.276ZINFOtest_unauthorized (dropshot): request completed latency_us = 5481 local_addr = [::1]:57786 method = GET remote_addr = [::1]:62454 req_id = 1c440ef8-203c-4857-9b6a-f86a4a7fee82 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
44772025-04-30T06:43:51.278ZTRACtest_unauthorized (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
44782025-04-30T06:43:51.279ZDEBGtest_unauthorized (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
44792025-04-30T06:43:51.279ZTRACtest_unauthorized (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
44802025-04-30T06:43:51.279ZTRACtest_unauthorized (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
44812025-04-30T06:43:51.279ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = http://[::1]:57786 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:57786/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "1c440ef8-203c-4857-9b6a-f86a4a7fee82", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
44822025-04-30T06:43:51.283ZDEBGtest_unauthorized (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
44832025-04-30T06:43:51.283ZDEBGtest_unauthorized (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
44842025-04-30T06:43:51.283ZDEBGtest_unauthorized (BackgroundTasks): finished collection background_task = inventory_collection
44852025-04-30T06:43:51.285ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
44862025-04-30T06:43:51.285ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
44872025-04-30T06:43:51.285ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
44882025-04-30T06:43:51.285ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
44892025-04-30T06:43:51.286ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
44902025-04-30T06:43:51.286ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
44912025-04-30T06:43:51.286ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
44922025-04-30T06:43:51.287ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
44932025-04-30T06:43:51.287ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 2 table_name = fields_string
44942025-04-30T06:43:51.287ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
44952025-04-30T06:43:51.287ZWARNtest_unauthorized (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
44962025-04-30T06:43:51.290ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44972025-04-30T06:43:51.295ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
44982025-04-30T06:43:51.295ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
44992025-04-30T06:43:51.296ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
45002025-04-30T06:43:51.296ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
45012025-04-30T06:43:51.300ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 1 table_name = measurements_i64
45022025-04-30T06:43:51.300ZTRACtest_unauthorized (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
45032025-04-30T06:43:51.338ZWARNtest_unauthorized (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
45042025-04-30T06:43:51.338ZWARNtest_unauthorized (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
45052025-04-30T06:43:51.338ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 545.461506ms iteration = 9 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
45062025-04-30T06:43:51.338ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 10 reason = Signaled
45072025-04-30T06:43:51.339ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
45082025-04-30T06:43:51.339ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
45092025-04-30T06:43:51.339ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
45102025-04-30T06:43:51.339ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
45112025-04-30T06:43:51.340ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
45122025-04-30T06:43:51.341ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
45132025-04-30T06:43:51.343ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45142025-04-30T06:43:51.346ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
45152025-04-30T06:43:51.347ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
45162025-04-30T06:43:51.347ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
45172025-04-30T06:43:51.348ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
45182025-04-30T06:43:51.351ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1a8b2318-abf8-406d-a389-112ab91a0de8, 0)]
45192025-04-30T06:43:51.351ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
45202025-04-30T06:43:51.352ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 41 a7 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 A............_mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:55161 req_id = 59596f42-3542-419a-97c0-2c51a5b027b5
45212025-04-30T06:43:51.352ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 16807,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:55161 req_id = 59596f42-3542-419a-97c0-2c51a5b027b5
45222025-04-30T06:43:51.353ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
45232025-04-30T06:43:51.353ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
45242025-04-30T06:43:51.353ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:55161 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 59596f42-3542-419a-97c0-2c51a5b027b5
45252025-04-30T06:43:51.354ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4296, tv_nsec: 152836824 } })
45262025-04-30T06:43:51.354ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
45272025-04-30T06:43:51.354ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: e9 c3 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:53447 req_id = ed541635-2db9-4f25-9e99-655ffd09ca19
45282025-04-30T06:43:51.354ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 59843,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:53447 req_id = ed541635-2db9-4f25-9e99-655ffd09ca19
45292025-04-30T06:43:51.354ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
45302025-04-30T06:43:51.354ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:53447 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = ed541635-2db9-4f25-9e99-655ffd09ca19
45312025-04-30T06:43:51.394ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
45322025-04-30T06:43:51.394ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
45332025-04-30T06:43:51.394ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
45342025-04-30T06:43:51.394ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
45352025-04-30T06:43:51.394ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
45362025-04-30T06:43:51.395ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
45372025-04-30T06:43:51.395ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 44 d2 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c D............_cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:61890 req_id = 93b64844-3bb0-4417-b073-2feec134b677
45382025-04-30T06:43:51.395ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 17618,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:61890 req_id = 93b64844-3bb0-4417-b073-2feec134b677
45392025-04-30T06:43:51.396ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
45402025-04-30T06:43:51.396ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 17618, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:61890 req_id = 93b64844-3bb0-4417-b073-2feec134b677
45412025-04-30T06:43:51.396ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
45422025-04-30T06:43:51.396ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
45432025-04-30T06:43:51.396ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
45442025-04-30T06:43:51.398ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45452025-04-30T06:43:51.403ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
45462025-04-30T06:43:51.403ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
45472025-04-30T06:43:51.403ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
45482025-04-30T06:43:51.403ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
45492025-04-30T06:43:51.403ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "f1c2706f-52ae-4c14-8440-43604cc72af5", "content-length": "835", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
45502025-04-30T06:43:51.404ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
45512025-04-30T06:43:51.406ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a0f18992-a58e-4fb7-82aa-ef0aee305151", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
45522025-04-30T06:43:51.406ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
45532025-04-30T06:43:51.407ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
45542025-04-30T06:43:51.407ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
45552025-04-30T06:43:51.407ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
45562025-04-30T06:43:51.409ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5f004427-9737-4d74-8a2a-9332910cdb19", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
45572025-04-30T06:43:51.409ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
45582025-04-30T06:43:51.411ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0f4a64f3-0ae1-4170-97bb-904fa37ae952", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
45592025-04-30T06:43:51.411ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
45602025-04-30T06:43:51.413ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c32a6f2e-9a11-4283-8235-bfa41e6e8120", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
45612025-04-30T06:43:51.413ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
45622025-04-30T06:43:51.416ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "282d6cba-7c1e-48e9-8a0d-6bd10ba69776", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
45632025-04-30T06:43:51.416ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
45642025-04-30T06:43:51.419ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2d3082f8-3392-454b-a131-6063a8474f77", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
45652025-04-30T06:43:51.419ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
45662025-04-30T06:43:51.421ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "adf5c0ec-4f11-47a3-8e21-31112ea313ee", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
45672025-04-30T06:43:51.421ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
45682025-04-30T06:43:51.422ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "30d85f0a-c60b-4d56-916d-ff06cb06ecb9", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
45692025-04-30T06:43:51.423ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
45702025-04-30T06:43:51.424ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "240a2b7d-9fb2-4843-9a81-c808d5f9b420", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
45712025-04-30T06:43:51.424ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
45722025-04-30T06:43:51.425ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "37d90687-35f7-4a43-8020-5f206c015238", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
45732025-04-30T06:43:51.426ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
45742025-04-30T06:43:51.427ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "875ea8f2-dcee-4183-8828-0d878965078a", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
45752025-04-30T06:43:51.427ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
45762025-04-30T06:43:51.429ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d2e50c89-bf3b-4502-9e48-513c1e9e49d9", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
45772025-04-30T06:43:51.429ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
45782025-04-30T06:43:51.434ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2bc171bb-2493-4c33-9586-b20fcd7bf427", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
45792025-04-30T06:43:51.434ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
45802025-04-30T06:43:51.437ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
45812025-04-30T06:43:51.440ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9cb45b5b-5b3f-4ad0-94ad-b6eb948c605d", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
45822025-04-30T06:43:51.440ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
45832025-04-30T06:43:51.442ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0626d625-a078-4a3d-963a-8e0bf44676d6", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
45842025-04-30T06:43:51.442ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
45852025-04-30T06:43:51.444ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b06d8b9b-04c4-4743-8a06-e52e3856b7ad", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
45862025-04-30T06:43:51.444ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
45872025-04-30T06:43:51.446ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb2c2734-43dc-4fee-9438-2d641cfb597c", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
45882025-04-30T06:43:51.446ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
45892025-04-30T06:43:51.447ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f2acac12-75cd-4a3b-92bd-d0fcf3e9974e", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
45902025-04-30T06:43:51.448ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
45912025-04-30T06:43:51.448ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9595873c-2ecd-45e2-b37c-01765b4f19e1", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
45922025-04-30T06:43:51.448ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
45932025-04-30T06:43:51.449ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "30957950-0031-4422-957a-cae5da45a3e2", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
45942025-04-30T06:43:51.450ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
45952025-04-30T06:43:51.450ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "658e31d3-57a9-4111-9a58-060bfe98a06b", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
45962025-04-30T06:43:51.451ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
45972025-04-30T06:43:51.451ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "bec8f0ca-33b1-49ac-9f65-7753f3faf02f", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
45982025-04-30T06:43:51.452ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
45992025-04-30T06:43:51.453ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "50cb0986-4345-4d45-9ef3-e5bf734670df", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
46002025-04-30T06:43:51.453ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
46012025-04-30T06:43:51.455ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4207ea1e-7536-4910-985f-ea579fd42140", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
46022025-04-30T06:43:51.456ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
46032025-04-30T06:43:51.457ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5439f2e5-8aaf-453e-9e41-fb2582681aa1", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
46042025-04-30T06:43:51.457ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
46052025-04-30T06:43:51.459ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "717e40de-cf8f-4fbb-ab10-013db18f3390", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
46062025-04-30T06:43:51.459ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
46072025-04-30T06:43:51.463ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a4290a59-7eb8-4f14-9d45-114c6d70ab63", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
46082025-04-30T06:43:51.463ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
46092025-04-30T06:43:51.467ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e1070c17-5606-4019-89ca-2fa4166750e7", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
46102025-04-30T06:43:51.467ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
46112025-04-30T06:43:51.469ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b8302995-b38a-41ef-a6cb-5eec62949687", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
46122025-04-30T06:43:51.469ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
46132025-04-30T06:43:51.471ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6848c015-4edd-4fbe-a87c-2516bef5a5ec", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
46142025-04-30T06:43:51.471ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
46152025-04-30T06:43:51.473ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "17013534-65fe-4daf-bc51-d28cadd846c4", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
46162025-04-30T06:43:51.474ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
46172025-04-30T06:43:51.476ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ab9aeb92-4663-4b0c-a4bb-153cd2797a91", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
46182025-04-30T06:43:51.476ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
46192025-04-30T06:43:51.477ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "792b5fe0-976b-4fb4-8edf-52da13ede707", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
46202025-04-30T06:43:51.477ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
46212025-04-30T06:43:51.479ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9a594569-cee4-4aa7-908e-4f6adc159464", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
46222025-04-30T06:43:51.479ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
46232025-04-30T06:43:51.481ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0e239fd9-0bd2-41ac-a152-236d11a7ad31", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
46242025-04-30T06:43:51.481ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
46252025-04-30T06:43:51.483ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "69bb0838-4638-4e06-b326-541646cf9d2a", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
46262025-04-30T06:43:51.483ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
46272025-04-30T06:43:51.487ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "727a8100-3675-4727-8146-89d4ca594059", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
46282025-04-30T06:43:51.488ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
46292025-04-30T06:43:51.488ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
46302025-04-30T06:43:51.488ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
46312025-04-30T06:43:51.488ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
46322025-04-30T06:43:51.488ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
46332025-04-30T06:43:51.489ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
46342025-04-30T06:43:51.492ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46352025-04-30T06:43:51.498ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
46362025-04-30T06:43:51.499ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
46372025-04-30T06:43:51.499ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
46382025-04-30T06:43:51.499ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
46392025-04-30T06:43:51.501ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9694456b-0ce7-4241-93b9-401eff30fe2b", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
46402025-04-30T06:43:51.501ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
46412025-04-30T06:43:51.502ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
46422025-04-30T06:43:51.502ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
46432025-04-30T06:43:51.502ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
46442025-04-30T06:43:51.505ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8d70e2a7-9dd8-4fef-8840-e6b7421b4f52", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
46452025-04-30T06:43:51.505ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
46462025-04-30T06:43:51.510ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a5b4e01-5269-4f47-b128-b957a8a5bff0", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
46472025-04-30T06:43:51.510ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
46482025-04-30T06:43:51.512ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "28bda6f0-dc4e-4c01-962e-59948571e20d", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
46492025-04-30T06:43:51.512ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
46502025-04-30T06:43:51.514ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "08f548e3-e546-441f-83de-1c83b6186e95", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
46512025-04-30T06:43:51.514ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
46522025-04-30T06:43:51.514ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cbec68c1-8ef4-4c43-b7ac-13c1e8d42f42", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
46532025-04-30T06:43:51.514ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
46542025-04-30T06:43:51.517ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1b3f6f72-63ab-4b9c-8b2f-04802ab952f4", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
46552025-04-30T06:43:51.517ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
46562025-04-30T06:43:51.518ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
46572025-04-30T06:43:51.518ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
46582025-04-30T06:43:51.518ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
46592025-04-30T06:43:51.518ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
46602025-04-30T06:43:51.520ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46612025-04-30T06:43:51.522ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
46622025-04-30T06:43:51.522ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
46632025-04-30T06:43:51.522ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
46642025-04-30T06:43:51.522ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
46652025-04-30T06:43:51.522ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
46662025-04-30T06:43:51.523ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
46672025-04-30T06:43:51.525ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46682025-04-30T06:43:51.530ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
46692025-04-30T06:43:51.530ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
46702025-04-30T06:43:51.530ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
46712025-04-30T06:43:51.531ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
46722025-04-30T06:43:51.586ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
46732025-04-30T06:43:51.586ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory
46742025-04-30T06:43:51.587ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:52500 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
46752025-04-30T06:43:51.588ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:52500 req_id = 69f12558-e5f2-4b91-8b92-a4d109bbe394 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
46762025-04-30T06:43:51.589ZINFOtest_unauthorized (dropshot): request completed latency_us = 826 local_addr = [::1]:57786 method = GET remote_addr = [::1]:52500 req_id = 69f12558-e5f2-4b91-8b92-a4d109bbe394 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
46772025-04-30T06:43:51.590ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = http://[::1]:57786 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:57786/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "69f12558-e5f2-4b91-8b92-a4d109bbe394", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
46782025-04-30T06:43:51.592ZDEBGtest_unauthorized (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
46792025-04-30T06:43:51.592ZDEBGtest_unauthorized (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
46802025-04-30T06:43:51.592ZDEBGtest_unauthorized (BackgroundTasks): finished collection background_task = inventory_collection
46812025-04-30T06:43:51.594ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
46822025-04-30T06:43:51.594ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
46832025-04-30T06:43:51.594ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
46842025-04-30T06:43:51.594ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
46852025-04-30T06:43:51.595ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
46862025-04-30T06:43:51.597ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46872025-04-30T06:43:51.601ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
46882025-04-30T06:43:51.601ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
46892025-04-30T06:43:51.601ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
46902025-04-30T06:43:51.602ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
46912025-04-30T06:43:51.621ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
46922025-04-30T06:43:51.646ZWARNtest_unauthorized (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
46932025-04-30T06:43:51.646ZWARNtest_unauthorized (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
46942025-04-30T06:43:51.647ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 308.138797ms iteration = 10 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
46952025-04-30T06:43:51.647ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 11 reason = Signaled
46962025-04-30T06:43:51.647ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
46972025-04-30T06:43:51.647ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
46982025-04-30T06:43:51.647ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
46992025-04-30T06:43:51.647ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
47002025-04-30T06:43:51.648ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
47012025-04-30T06:43:51.651ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47022025-04-30T06:43:51.659ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
47032025-04-30T06:43:51.659ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
47042025-04-30T06:43:51.659ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
47052025-04-30T06:43:51.660ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
47062025-04-30T06:43:51.663ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1a8b2318-abf8-406d-a389-112ab91a0de8, 0)]
47072025-04-30T06:43:51.663ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
47082025-04-30T06:43:51.664ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: f7 54 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .T..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:51915 req_id = f4aa46c2-9027-492b-a6a6-97f07fbaea7d
47092025-04-30T06:43:51.664ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 63316,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51915 req_id = f4aa46c2-9027-492b-a6a6-97f07fbaea7d
47102025-04-30T06:43:51.664ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
47112025-04-30T06:43:51.665ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
47122025-04-30T06:43:51.665ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:51915 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = f4aa46c2-9027-492b-a6a6-97f07fbaea7d
47132025-04-30T06:43:51.666ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4296, tv_nsec: 465029508 } })
47142025-04-30T06:43:51.666ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
47152025-04-30T06:43:51.667ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: e8 3a 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .:..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:59237 req_id = 709844a0-4e2b-4100-b638-4364d79366cc
47162025-04-30T06:43:51.667ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 59450,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:59237 req_id = 709844a0-4e2b-4100-b638-4364d79366cc
47172025-04-30T06:43:51.668ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
47182025-04-30T06:43:51.668ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:59237 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 709844a0-4e2b-4100-b638-4364d79366cc
47192025-04-30T06:43:51.717ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
47202025-04-30T06:43:51.717ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
47212025-04-30T06:43:51.717ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
47222025-04-30T06:43:51.717ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
47232025-04-30T06:43:51.717ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
47242025-04-30T06:43:51.718ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
47252025-04-30T06:43:51.718ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 6e bd 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c n............_cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:59736 req_id = c2a66010-8a7d-4ca0-8c94-f72fd42caf73
47262025-04-30T06:43:51.718ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 28349,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:59736 req_id = c2a66010-8a7d-4ca0-8c94-f72fd42caf73
47272025-04-30T06:43:51.719ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
47282025-04-30T06:43:51.719ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 28349, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:59736 req_id = c2a66010-8a7d-4ca0-8c94-f72fd42caf73
47292025-04-30T06:43:51.719ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
47302025-04-30T06:43:51.719ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
47312025-04-30T06:43:51.719ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
47322025-04-30T06:43:51.721ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47332025-04-30T06:43:51.728ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
47342025-04-30T06:43:51.728ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
47352025-04-30T06:43:51.728ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
47362025-04-30T06:43:51.728ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
47372025-04-30T06:43:51.731ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "566f65b0-ef21-4b3d-a679-32f63b5e8b71", "content-length": "835", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
47382025-04-30T06:43:51.731ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
47392025-04-30T06:43:51.731ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
47402025-04-30T06:43:51.731ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
47412025-04-30T06:43:51.732ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
47422025-04-30T06:43:51.734ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d2a0dcbd-6c5f-4842-a644-64f4b9e8b0cd", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
47432025-04-30T06:43:51.735ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
47442025-04-30T06:43:51.737ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b17be756-688f-41dd-8958-87e40e6844d7", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
47452025-04-30T06:43:51.737ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
47462025-04-30T06:43:51.741ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "eb23e8de-c907-4b43-b1bf-28fb10c2e885", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
47472025-04-30T06:43:51.741ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
47482025-04-30T06:43:51.743ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fc3a477e-e18a-4999-aded-3705cd663d4b", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
47492025-04-30T06:43:51.743ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
47502025-04-30T06:43:51.746ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "05eae111-7cfe-4d23-92ec-96a6b13d8c60", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
47512025-04-30T06:43:51.746ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
47522025-04-30T06:43:51.749ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f6ff1ed5-a8d0-438d-acae-93893d6c4f6a", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
47532025-04-30T06:43:51.749ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
47542025-04-30T06:43:51.752ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fef09a4a-57bc-4755-88e6-13aa496025f2", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
47552025-04-30T06:43:51.752ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
47562025-04-30T06:43:51.753ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7d16a1bc-6b52-40f7-93f7-8688b8b3b938", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
47572025-04-30T06:43:51.753ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
47582025-04-30T06:43:51.755ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ed23a388-3d4a-4ca3-9fbd-9297a8323712", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
47592025-04-30T06:43:51.755ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
47602025-04-30T06:43:51.756ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e9182e8e-b5ab-48b1-9341-8b6cb30fa678", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
47612025-04-30T06:43:51.756ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
47622025-04-30T06:43:51.758ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "de354095-608e-48bd-a68f-cd3cc67ceecc", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
47632025-04-30T06:43:51.758ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
47642025-04-30T06:43:51.759ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9e58847e-43a7-4c53-9a38-230a1bcc2a2b", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
47652025-04-30T06:43:51.760ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
47662025-04-30T06:43:51.763ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
47672025-04-30T06:43:51.765ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "aee0f381-07a6-4d42-92e7-7746b16a974f", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
47682025-04-30T06:43:51.766ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
47692025-04-30T06:43:51.767ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d955c505-8e13-4bb6-9b64-af61ea5e3b04", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
47702025-04-30T06:43:51.768ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
47712025-04-30T06:43:51.770ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f830237e-0efd-48ce-b9b2-56958b97b5bf", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
47722025-04-30T06:43:51.770ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
47732025-04-30T06:43:51.772ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "99654ab8-eee0-42c9-89a2-fbbf5d8c20c1", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
47742025-04-30T06:43:51.772ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
47752025-04-30T06:43:51.774ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cb66db86-c532-4e07-a1b3-f401c6c02a1e", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
47762025-04-30T06:43:51.774ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
47772025-04-30T06:43:51.775ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7587af6c-dd06-4fa2-83c1-73757552c7cb", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
47782025-04-30T06:43:51.776ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
47792025-04-30T06:43:51.778ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d849258c-b90f-4d6d-8521-522668e618f8", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
47802025-04-30T06:43:51.781ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
47812025-04-30T06:43:51.783ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e68ddf1f-cb5b-4104-9130-8e87fadedbbf", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
47822025-04-30T06:43:51.783ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
47832025-04-30T06:43:51.785ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "64eb5da9-3381-4a08-aca6-33e4413cab8f", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
47842025-04-30T06:43:51.785ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
47852025-04-30T06:43:51.787ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e87d1056-998a-4cb2-b231-49dc406f114a", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
47862025-04-30T06:43:51.788ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
47872025-04-30T06:43:51.793ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7fd20df5-3807-4aef-bdea-29621eada923", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
47882025-04-30T06:43:51.793ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
47892025-04-30T06:43:51.801ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "10e3ead5-848e-4c17-b981-d77c11306c6e", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
47902025-04-30T06:43:51.801ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
47912025-04-30T06:43:51.803ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e7f02153-c076-47bb-a90f-8a80a01ca6ce", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
47922025-04-30T06:43:51.804ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
47932025-04-30T06:43:51.806ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e064cb2a-06b0-4b4b-8757-158d210f65ed", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
47942025-04-30T06:43:51.806ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
47952025-04-30T06:43:51.813ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "597cab2e-f65b-4fbf-a853-e22e7916db72", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
47962025-04-30T06:43:51.813ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
47972025-04-30T06:43:51.814ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
47982025-04-30T06:43:51.814ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
47992025-04-30T06:43:51.814ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
48002025-04-30T06:43:51.814ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
48012025-04-30T06:43:51.820ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
48022025-04-30T06:43:51.822ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "40603487-4abe-43a2-8ffc-468f2df8d0dd", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
48032025-04-30T06:43:51.822ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
48042025-04-30T06:43:51.823ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48052025-04-30T06:43:51.831ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
48062025-04-30T06:43:51.831ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
48072025-04-30T06:43:51.831ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
48082025-04-30T06:43:51.831ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
48092025-04-30T06:43:51.854ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "71295f30-3d91-4218-a08d-c9d29f83a726", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
48102025-04-30T06:43:51.854ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
48112025-04-30T06:43:51.855ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
48122025-04-30T06:43:51.855ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
48132025-04-30T06:43:51.856ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
48142025-04-30T06:43:51.858ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b42c4258-1bd2-4d5e-99cf-0e0772a6aa9b", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
48152025-04-30T06:43:51.861ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
48162025-04-30T06:43:51.862ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8ae7b894-3982-4633-bc43-b105b6709124", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
48172025-04-30T06:43:51.862ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
48182025-04-30T06:43:51.867ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "03db2656-fbb2-4595-b4e6-b0a7689e5cb5", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
48192025-04-30T06:43:51.867ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
48202025-04-30T06:43:51.872ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "668787f0-a680-4575-8c57-9aed14292fbe", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
48212025-04-30T06:43:51.873ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
48222025-04-30T06:43:51.875ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2cebc54b-14e4-49a8-a34f-02ce8920e664", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
48232025-04-30T06:43:51.875ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
48242025-04-30T06:43:51.880ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "702a206e-2f0a-4116-9278-12a98777d4d1", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
48252025-04-30T06:43:51.880ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
48262025-04-30T06:43:51.891ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5bee96d3-da4a-473f-aeee-b1fb58ab6c4c", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
48272025-04-30T06:43:51.891ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
48282025-04-30T06:43:51.902ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e21c98f8-277c-4cae-add4-50719c426cf6", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
48292025-04-30T06:43:51.902ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
48302025-04-30T06:43:51.906ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3d531036-0ecc-489f-8da2-40e6bb5e9108", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
48312025-04-30T06:43:51.907ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
48322025-04-30T06:43:51.910ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
48332025-04-30T06:43:51.912ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4a2dc4ee-d125-4f42-9853-169858415f7f", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
48342025-04-30T06:43:51.912ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
48352025-04-30T06:43:51.916ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "44b331b8-1530-44c6-b54d-5c9343cf06a6", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
48362025-04-30T06:43:51.916ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
48372025-04-30T06:43:51.917ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "bf2fa027-8793-44ea-bba3-fb69e2869074", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
48382025-04-30T06:43:51.917ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
48392025-04-30T06:43:51.922ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "19194ef4-c978-4d52-9f9b-10872dad2880", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
48402025-04-30T06:43:51.923ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
48412025-04-30T06:43:51.925ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c7fa64d1-d08a-4533-a99d-b6973166154e", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
48422025-04-30T06:43:51.925ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
48432025-04-30T06:43:51.927ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d5fe654c-a488-481b-a441-2b634883ad18", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
48442025-04-30T06:43:51.927ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
48452025-04-30T06:43:51.927ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
48462025-04-30T06:43:51.927ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
48472025-04-30T06:43:51.927ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
48482025-04-30T06:43:51.928ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
48492025-04-30T06:43:51.943ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48502025-04-30T06:43:51.949ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
48512025-04-30T06:43:51.949ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
48522025-04-30T06:43:51.949ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
48532025-04-30T06:43:51.949ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
48542025-04-30T06:43:51.950ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
48552025-04-30T06:43:51.950ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
48562025-04-30T06:43:51.963ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
48572025-04-30T06:43:51.963ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
48582025-04-30T06:43:51.963ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
48592025-04-30T06:43:51.963ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
48602025-04-30T06:43:51.964ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
48612025-04-30T06:43:51.965ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48622025-04-30T06:43:51.968ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
48632025-04-30T06:43:51.968ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
48642025-04-30T06:43:51.969ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
48652025-04-30T06:43:51.969ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
48662025-04-30T06:43:51.981ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48672025-04-30T06:43:51.986ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
48682025-04-30T06:43:51.986ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
48692025-04-30T06:43:51.986ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
48702025-04-30T06:43:51.987ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
48712025-04-30T06:43:52.062ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
48722025-04-30T06:43:52.062ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory
48732025-04-30T06:43:52.063ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
48742025-04-30T06:43:52.063ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
48752025-04-30T06:43:52.063ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
48762025-04-30T06:43:52.064ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:51603 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
48772025-04-30T06:43:52.067ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:51603 req_id = 1a8f08d4-8ecb-4526-81a2-fe7c153bdcef sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
48782025-04-30T06:43:52.068ZINFOtest_unauthorized (dropshot): request completed latency_us = 644 local_addr = [::1]:57786 method = GET remote_addr = [::1]:51603 req_id = 1a8f08d4-8ecb-4526-81a2-fe7c153bdcef response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
48792025-04-30T06:43:52.069ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = http://[::1]:57786 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:57786/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a8f08d4-8ecb-4526-81a2-fe7c153bdcef", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
48802025-04-30T06:43:52.071ZDEBGtest_unauthorized (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
48812025-04-30T06:43:52.071ZDEBGtest_unauthorized (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
48822025-04-30T06:43:52.071ZDEBGtest_unauthorized (BackgroundTasks): finished collection background_task = inventory_collection
48832025-04-30T06:43:52.074ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
48842025-04-30T06:43:52.074ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
48852025-04-30T06:43:52.075ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
48862025-04-30T06:43:52.075ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
48872025-04-30T06:43:52.076ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
48882025-04-30T06:43:52.081ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48892025-04-30T06:43:52.086ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
48902025-04-30T06:43:52.086ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
48912025-04-30T06:43:52.086ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
48922025-04-30T06:43:52.087ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
48932025-04-30T06:43:52.111ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
48942025-04-30T06:43:52.119ZWARNtest_unauthorized (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
48952025-04-30T06:43:52.119ZWARNtest_unauthorized (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
48962025-04-30T06:43:52.119ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 472.678931ms iteration = 11 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
48972025-04-30T06:43:52.119ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 12 reason = Signaled
48982025-04-30T06:43:52.120ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
48992025-04-30T06:43:52.120ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
49002025-04-30T06:43:52.120ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
49012025-04-30T06:43:52.120ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
49022025-04-30T06:43:52.122ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
49032025-04-30T06:43:52.123ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49042025-04-30T06:43:52.127ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
49052025-04-30T06:43:52.127ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
49062025-04-30T06:43:52.127ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
49072025-04-30T06:43:52.127ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
49082025-04-30T06:43:52.131ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1a8b2318-abf8-406d-a389-112ab91a0de8, 0)]
49092025-04-30T06:43:52.131ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
49102025-04-30T06:43:52.134ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 83 36 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .6..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:59030 req_id = e2b10ea4-2259-4715-a84f-360f0ae61347
49112025-04-30T06:43:52.134ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 33590,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:59030 req_id = e2b10ea4-2259-4715-a84f-360f0ae61347
49122025-04-30T06:43:52.135ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
49132025-04-30T06:43:52.135ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
49142025-04-30T06:43:52.135ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:59030 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = e2b10ea4-2259-4715-a84f-360f0ae61347
49152025-04-30T06:43:52.136ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4296, tv_nsec: 935286975 } })
49162025-04-30T06:43:52.136ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
49172025-04-30T06:43:52.136ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 74 39 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e t9..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:62570 req_id = 0c8c9313-5bf7-499d-add1-78f258677545
49182025-04-30T06:43:52.136ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 29753,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62570 req_id = 0c8c9313-5bf7-499d-add1-78f258677545
49192025-04-30T06:43:52.137ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
49202025-04-30T06:43:52.137ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:62570 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 0c8c9313-5bf7-499d-add1-78f258677545
49212025-04-30T06:43:52.185ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
49222025-04-30T06:43:52.185ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
49232025-04-30T06:43:52.185ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
49242025-04-30T06:43:52.185ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
49252025-04-30T06:43:52.185ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
49262025-04-30T06:43:52.186ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
49272025-04-30T06:43:52.186ZDEBGtest_unauthorized (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
49282025-04-30T06:43:52.186ZDEBGtest_unauthorized (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
49292025-04-30T06:43:52.186ZDEBGtest_unauthorized (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a
49302025-04-30T06:43:52.187ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
49312025-04-30T06:43:52.188ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: d6 f5 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:64323 req_id = 81b65f95-02d6-4c43-9cb4-851f5886736c
49322025-04-30T06:43:52.189ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 55029,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64323 req_id = 81b65f95-02d6-4c43-9cb4-851f5886736c
49332025-04-30T06:43:52.189ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
49342025-04-30T06:43:52.189ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 55029, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:64323 req_id = 81b65f95-02d6-4c43-9cb4-851f5886736c
49352025-04-30T06:43:52.190ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49362025-04-30T06:43:52.195ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
49372025-04-30T06:43:52.195ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
49382025-04-30T06:43:52.195ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
49392025-04-30T06:43:52.196ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
49402025-04-30T06:43:52.197ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
49412025-04-30T06:43:52.197ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
49422025-04-30T06:43:52.197ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
49432025-04-30T06:43:52.201ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 2 table_name = fields_string
49442025-04-30T06:43:52.201ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
49452025-04-30T06:43:52.202ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
49462025-04-30T06:43:52.202ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
49472025-04-30T06:43:52.202ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
49482025-04-30T06:43:52.203ZDEBGtest_unauthorized (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
49492025-04-30T06:43:52.203ZDEBGtest_unauthorized (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
49502025-04-30T06:43:52.208ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 1 table_name = measurements_i64
49512025-04-30T06:43:52.208ZTRACtest_unauthorized (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
49522025-04-30T06:43:52.209ZTRACtest_unauthorized (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
49532025-04-30T06:43:52.210ZDEBGtest_unauthorized (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
49542025-04-30T06:43:52.211ZTRACtest_unauthorized (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
49552025-04-30T06:43:52.211ZTRACtest_unauthorized (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
49562025-04-30T06:43:52.212ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "d2839c34-9f2f-4372-ae0b-fa0334fc1ece", "content-length": "835", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
49572025-04-30T06:43:52.212ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
49582025-04-30T06:43:52.214ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2168aa75-8ed9-45cb-a935-dbd867383a61", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
49592025-04-30T06:43:52.215ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
49602025-04-30T06:43:52.222ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9f4ce02c-dd2c-49f0-9421-a0a5ec3d7227", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
49612025-04-30T06:43:52.222ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
49622025-04-30T06:43:52.228ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6e3d516e-ea80-4f96-ae07-ac67a5c2bf2b", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
49632025-04-30T06:43:52.228ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
49642025-04-30T06:43:52.240ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d43933ea-5581-4c9f-b70d-3ecae5949f89", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
49652025-04-30T06:43:52.240ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
49662025-04-30T06:43:52.250ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8f0badd0-e8ef-42da-9f50-ef26b5b97baf", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
49672025-04-30T06:43:52.250ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
49682025-04-30T06:43:52.259ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fddfed81-a144-425a-bcad-563ebff58f56", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
49692025-04-30T06:43:52.260ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
49702025-04-30T06:43:52.260ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
49712025-04-30T06:43:52.263ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c5bd24c3-465e-4fe7-9d62-f19cd4186a27", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
49722025-04-30T06:43:52.263ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
49732025-04-30T06:43:52.265ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "76983511-6497-48a1-904e-c14ae00fbdb1", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
49742025-04-30T06:43:52.265ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
49752025-04-30T06:43:52.268ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c910d467-10ee-4b79-98b8-9c30997cdce0", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
49762025-04-30T06:43:52.268ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
49772025-04-30T06:43:52.269ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a2b8cd90-f2af-464a-8e22-76cd74428f06", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
49782025-04-30T06:43:52.270ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
49792025-04-30T06:43:52.271ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5136615b-cfc3-42cf-9bd6-2e0c3bb5868e", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
49802025-04-30T06:43:52.271ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
49812025-04-30T06:43:52.273ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d2c0934d-50da-4acb-a15d-32e31ce80c3d", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
49822025-04-30T06:43:52.273ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
49832025-04-30T06:43:52.275ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1fa466d3-14d7-4fff-8369-e5c63f66b724", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
49842025-04-30T06:43:52.275ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
49852025-04-30T06:43:52.278ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c2e33e20-5229-442a-9006-2e0b64967a42", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
49862025-04-30T06:43:52.278ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
49872025-04-30T06:43:52.279ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "454c676a-46a8-4933-ae9a-6282f6dbd0e9", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
49882025-04-30T06:43:52.280ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
49892025-04-30T06:43:52.283ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "183e8427-8719-490a-a4a9-98647a3d4c75", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
49902025-04-30T06:43:52.283ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
49912025-04-30T06:43:52.285ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e023d979-8e4d-4fb5-9344-37c96274a848", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
49922025-04-30T06:43:52.285ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
49932025-04-30T06:43:52.287ZWARNtest_unauthorized (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
49942025-04-30T06:43:52.288ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4bf5e471-9f0c-40c2-8c39-7781ea3aca8a", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
49952025-04-30T06:43:52.288ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
49962025-04-30T06:43:52.289ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "09b9468c-40dc-4fc6-9d8d-ed76be393c26", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
49972025-04-30T06:43:52.289ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
49982025-04-30T06:43:52.290ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "73db0a33-0ba3-4d98-95fd-57f8ccac2dd9", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
49992025-04-30T06:43:52.290ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
50002025-04-30T06:43:52.292ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ea630492-2e50-42a9-a98c-898936f13626", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
50012025-04-30T06:43:52.292ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
50022025-04-30T06:43:52.293ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cb843ef4-bbec-420e-b13d-ed7e4cda3099", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
50032025-04-30T06:43:52.293ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
50042025-04-30T06:43:52.295ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1f9cfe5b-831d-4302-b5fc-aed90d3cdc85", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
50052025-04-30T06:43:52.295ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
50062025-04-30T06:43:52.298ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e1451449-a917-447d-ab16-1bc9d7eef72a", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
50072025-04-30T06:43:52.298ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
50082025-04-30T06:43:52.300ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d683c4b3-ae1f-472f-a12f-431fbe845b48", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
50092025-04-30T06:43:52.301ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
50102025-04-30T06:43:52.303ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e31cc801-c12b-4bdb-b965-21adf332e90f", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
50112025-04-30T06:43:52.303ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
50122025-04-30T06:43:52.306ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f9622690-29af-45f0-b9ce-aa4287854a54", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
50132025-04-30T06:43:52.306ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
50142025-04-30T06:43:52.308ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6fb669b7-beb7-47db-857c-b191295f8fac", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
50152025-04-30T06:43:52.308ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
50162025-04-30T06:43:52.311ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
50172025-04-30T06:43:52.311ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
50182025-04-30T06:43:52.311ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
50192025-04-30T06:43:52.311ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
50202025-04-30T06:43:52.313ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
50212025-04-30T06:43:52.314ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c21295fe-e9a0-4d35-879f-5bc5684f8627", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
50222025-04-30T06:43:52.314ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
50232025-04-30T06:43:52.316ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f02aa7d5-4e28-4331-bc09-90474a86e891", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
50242025-04-30T06:43:52.316ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
50252025-04-30T06:43:52.318ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "21bb13b5-bf3e-4d20-8de7-f891c41dea68", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
50262025-04-30T06:43:52.318ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
50272025-04-30T06:43:52.334ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50282025-04-30T06:43:52.340ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
50292025-04-30T06:43:52.341ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
50302025-04-30T06:43:52.341ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
50312025-04-30T06:43:52.341ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
50322025-04-30T06:43:52.342ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "da364bb6-efff-4011-85d7-15d7c29838f3", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
50332025-04-30T06:43:52.342ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
50342025-04-30T06:43:52.344ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "662c4d02-7663-450d-bb73-914937a42cf7", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
50352025-04-30T06:43:52.344ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
50362025-04-30T06:43:52.344ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
50372025-04-30T06:43:52.344ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
50382025-04-30T06:43:52.345ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
50392025-04-30T06:43:52.346ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a9897c6-9499-4db4-9347-ff12664699f5", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
50402025-04-30T06:43:52.346ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
50412025-04-30T06:43:52.349ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "78eab975-2dbc-4943-9990-bd94897ff6ac", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
50422025-04-30T06:43:52.349ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
50432025-04-30T06:43:52.352ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a983e8f3-2a27-4f7b-b93a-644e32745405", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
50442025-04-30T06:43:52.352ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
50452025-04-30T06:43:52.353ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b9e048b9-bd5b-4bbb-a150-f5f308ae5082", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
50462025-04-30T06:43:52.354ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
50472025-04-30T06:43:52.356ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fd0324dc-ee23-4563-a4ec-c9ca4ff9d021", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
50482025-04-30T06:43:52.356ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
50492025-04-30T06:43:52.358ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a7deb34d-b897-4a76-b486-68be17515665", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
50502025-04-30T06:43:52.358ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
50512025-04-30T06:43:52.360ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bc2a3343-117d-4671-9837-b7bbab20fb37", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
50522025-04-30T06:43:52.360ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
50532025-04-30T06:43:52.361ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "bf971573-4d6a-42b9-b23e-265c92ae4d37", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
50542025-04-30T06:43:52.361ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
50552025-04-30T06:43:52.362ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3d9924e6-5697-4046-abda-9b8a84f1b71d", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
50562025-04-30T06:43:52.362ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
50572025-04-30T06:43:52.363ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a2f6dd82-4ccf-4e03-9b9a-157369ac3a47", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
50582025-04-30T06:43:52.364ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
50592025-04-30T06:43:52.364ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1f29ebba-1f00-4d2c-b8ae-f5078f917b04", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
50602025-04-30T06:43:52.365ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
50612025-04-30T06:43:52.365ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
50622025-04-30T06:43:52.365ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
50632025-04-30T06:43:52.365ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
50642025-04-30T06:43:52.365ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
50652025-04-30T06:43:52.368ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50662025-04-30T06:43:52.371ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
50672025-04-30T06:43:52.372ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
50682025-04-30T06:43:52.372ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
50692025-04-30T06:43:52.372ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
50702025-04-30T06:43:52.372ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
50712025-04-30T06:43:52.372ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
50722025-04-30T06:43:52.375ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50732025-04-30T06:43:52.380ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
50742025-04-30T06:43:52.380ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
50752025-04-30T06:43:52.380ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
50762025-04-30T06:43:52.380ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
50772025-04-30T06:43:52.433ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
50782025-04-30T06:43:52.433ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory
50792025-04-30T06:43:52.434ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
50802025-04-30T06:43:52.435ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:44306 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
50812025-04-30T06:43:52.437ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:44306 req_id = 3add1bb7-e48c-49e6-b768-8fdbc7419a4d sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
50822025-04-30T06:43:52.437ZINFOtest_unauthorized (dropshot): request completed latency_us = 656 local_addr = [::1]:57786 method = GET remote_addr = [::1]:44306 req_id = 3add1bb7-e48c-49e6-b768-8fdbc7419a4d response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
50832025-04-30T06:43:52.438ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = http://[::1]:57786 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:57786/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "3add1bb7-e48c-49e6-b768-8fdbc7419a4d", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
50842025-04-30T06:43:52.444ZDEBGtest_unauthorized (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
50852025-04-30T06:43:52.444ZDEBGtest_unauthorized (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
50862025-04-30T06:43:52.444ZDEBGtest_unauthorized (BackgroundTasks): finished collection background_task = inventory_collection
50872025-04-30T06:43:52.449ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
50882025-04-30T06:43:52.449ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
50892025-04-30T06:43:52.450ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
50902025-04-30T06:43:52.450ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
50912025-04-30T06:43:52.450ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
50922025-04-30T06:43:52.454ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50932025-04-30T06:43:52.460ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
50942025-04-30T06:43:52.460ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
50952025-04-30T06:43:52.460ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
50962025-04-30T06:43:52.461ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
50972025-04-30T06:43:52.486ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
50982025-04-30T06:43:52.486ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
50992025-04-30T06:43:52.486ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
51002025-04-30T06:43:52.486ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
51012025-04-30T06:43:52.487ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
51022025-04-30T06:43:52.498ZWARNtest_unauthorized (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
51032025-04-30T06:43:52.498ZWARNtest_unauthorized (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
51042025-04-30T06:43:52.498ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 378.640677ms iteration = 12 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
51052025-04-30T06:43:52.498ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 13 reason = Signaled
51062025-04-30T06:43:52.498ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
51072025-04-30T06:43:52.498ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
51082025-04-30T06:43:52.498ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
51092025-04-30T06:43:52.498ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
51102025-04-30T06:43:52.500ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
51112025-04-30T06:43:52.500ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51122025-04-30T06:43:52.509ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
51132025-04-30T06:43:52.509ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
51142025-04-30T06:43:52.509ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
51152025-04-30T06:43:52.510ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
51162025-04-30T06:43:52.516ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51172025-04-30T06:43:52.524ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
51182025-04-30T06:43:52.524ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
51192025-04-30T06:43:52.524ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
51202025-04-30T06:43:52.524ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
51212025-04-30T06:43:52.530ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
51222025-04-30T06:43:52.530ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
51232025-04-30T06:43:52.530ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
51242025-04-30T06:43:52.531ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1a8b2318-abf8-406d-a389-112ab91a0de8, 0)]
51252025-04-30T06:43:52.531ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
51262025-04-30T06:43:52.536ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: ee 04 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:64733 req_id = 1c011d3d-f24b-4ad0-a037-8f690ccb4769
51272025-04-30T06:43:52.536ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 60932,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64733 req_id = 1c011d3d-f24b-4ad0-a037-8f690ccb4769
51282025-04-30T06:43:52.536ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
51292025-04-30T06:43:52.541ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
51302025-04-30T06:43:52.541ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:64733 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 1c011d3d-f24b-4ad0-a037-8f690ccb4769
51312025-04-30T06:43:52.546ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4297, tv_nsec: 343082721 } })
51322025-04-30T06:43:52.546ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
51332025-04-30T06:43:52.548ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 2f 47 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e /G..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:60256 req_id = 49630614-33af-4187-9598-3e744fcf700f
51342025-04-30T06:43:52.549ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 12103,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60256 req_id = 49630614-33af-4187-9598-3e744fcf700f
51352025-04-30T06:43:52.549ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
51362025-04-30T06:43:52.549ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:60256 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 49630614-33af-4187-9598-3e744fcf700f
51372025-04-30T06:43:52.639ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
51382025-04-30T06:43:52.646ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 15 d6 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:60972 req_id = 87e68a41-0945-4513-96a0-e3311421ec5b
51392025-04-30T06:43:52.646ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 5590,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60972 req_id = 87e68a41-0945-4513-96a0-e3311421ec5b
51402025-04-30T06:43:52.646ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
51412025-04-30T06:43:52.646ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 5590, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:60972 req_id = 87e68a41-0945-4513-96a0-e3311421ec5b
51422025-04-30T06:43:52.649ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
51432025-04-30T06:43:52.649ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
51442025-04-30T06:43:52.649ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
51452025-04-30T06:43:52.663ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd055144-bb5e-404c-8e77-a2f1a5f88ff6", "content-length": "835", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
51462025-04-30T06:43:52.664ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
51472025-04-30T06:43:52.676ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5c83388e-726b-43f8-894a-06631b86614b", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
51482025-04-30T06:43:52.677ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
51492025-04-30T06:43:52.680ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "36908eeb-bd7b-4cbd-a328-d1bfa52e69c4", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
51502025-04-30T06:43:52.680ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
51512025-04-30T06:43:52.683ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "aa780c15-8845-4985-8d47-2081357fb49b", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
51522025-04-30T06:43:52.683ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
51532025-04-30T06:43:52.686ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
51542025-04-30T06:43:52.688ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b6b950df-1945-4023-8b1e-f624256a1d51", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
51552025-04-30T06:43:52.688ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
51562025-04-30T06:43:52.691ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2481f688-4675-4c3d-8dd5-ffef852c1a37", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
51572025-04-30T06:43:52.691ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
51582025-04-30T06:43:52.695ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a9c1eee-4b68-4a50-bbd3-fd030114d968", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
51592025-04-30T06:43:52.695ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
51602025-04-30T06:43:52.700ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "47c1d3a2-d294-42fc-b3f7-24e4886d778d", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
51612025-04-30T06:43:52.700ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
51622025-04-30T06:43:52.703ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "93dc5fe6-91ee-4b7c-af41-d7c37c5d5616", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
51632025-04-30T06:43:52.703ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
51642025-04-30T06:43:52.704ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0802fde-7674-455f-acaf-d584989f7f94", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
51652025-04-30T06:43:52.704ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
51662025-04-30T06:43:52.705ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "13d86d29-c67d-4291-a38e-eec8924c2c7c", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
51672025-04-30T06:43:52.705ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
51682025-04-30T06:43:52.706ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6c624b06-f414-495a-9c06-3a3047e7183d", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
51692025-04-30T06:43:52.706ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
51702025-04-30T06:43:52.708ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "866a3465-dc0c-4c70-a6a0-b093efd46c48", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
51712025-04-30T06:43:52.708ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
51722025-04-30T06:43:52.710ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "102c3578-2f22-44bb-9da8-80a2538cd02d", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
51732025-04-30T06:43:52.710ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
51742025-04-30T06:43:52.717ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "dcaf78a1-0bd5-4ce6-9b3d-0b51ee1c4c55", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
51752025-04-30T06:43:52.717ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
51762025-04-30T06:43:52.720ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bd6d497d-2fcd-4298-b504-406ad7e41bea", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
51772025-04-30T06:43:52.720ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
51782025-04-30T06:43:52.722ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d04d0b0f-bc6c-4135-b76d-d9083a5ab9d5", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
51792025-04-30T06:43:52.722ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
51802025-04-30T06:43:52.724ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f4fd3170-6280-4fb5-98f7-958510df191d", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
51812025-04-30T06:43:52.724ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
51822025-04-30T06:43:52.726ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "40a7ad86-b063-4977-9ac3-7b8397a31011", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
51832025-04-30T06:43:52.726ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
51842025-04-30T06:43:52.727ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c1518c94-73e2-4f78-a2d2-34e515c9d0a4", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
51852025-04-30T06:43:52.727ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
51862025-04-30T06:43:52.728ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a4f8a482-797a-425e-925e-d4b5a7c718c5", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
51872025-04-30T06:43:52.728ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
51882025-04-30T06:43:52.730ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c9e840c0-dc93-4742-8c69-eade1434d7cc", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
51892025-04-30T06:43:52.730ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
51902025-04-30T06:43:52.731ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "11531161-4ac3-4307-addb-f70ded3de6cc", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
51912025-04-30T06:43:52.731ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
51922025-04-30T06:43:52.732ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cec624e1-37da-4ac3-b7f4-8dd9b3887b8a", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
51932025-04-30T06:43:52.732ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
51942025-04-30T06:43:52.734ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "35acf316-e948-4df5-bc5a-eb00407dd637", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
51952025-04-30T06:43:52.735ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
51962025-04-30T06:43:52.737ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
51972025-04-30T06:43:52.737ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
51982025-04-30T06:43:52.737ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
51992025-04-30T06:43:52.737ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
52002025-04-30T06:43:52.738ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
52012025-04-30T06:43:52.741ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8bedeb41-7b0e-4c5d-bc8a-352742e44a7d", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
52022025-04-30T06:43:52.741ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
52032025-04-30T06:43:52.742ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52042025-04-30T06:43:52.747ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
52052025-04-30T06:43:52.747ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
52062025-04-30T06:43:52.747ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
52072025-04-30T06:43:52.747ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
52082025-04-30T06:43:52.750ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "126b75eb-e21f-4cf7-9e21-300b8b2d6dd1", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
52092025-04-30T06:43:52.750ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
52102025-04-30T06:43:52.750ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
52112025-04-30T06:43:52.750ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
52122025-04-30T06:43:52.750ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
52132025-04-30T06:43:52.753ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "71474e21-e072-42a2-9f3b-476cf2afab82", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
52142025-04-30T06:43:52.757ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
52152025-04-30T06:43:52.760ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9ef36cc5-9ef5-4c9e-bfd5-2e89a80bf916", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
52162025-04-30T06:43:52.760ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
52172025-04-30T06:43:52.763ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cb798657-13ca-4c13-94ff-6f8253cbcfce", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
52182025-04-30T06:43:52.763ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
52192025-04-30T06:43:52.764ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "16c1e53c-0fce-4a81-a7f7-842990bead4a", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
52202025-04-30T06:43:52.764ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
52212025-04-30T06:43:52.765ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "539730e1-6320-419f-9407-06514e8756ce", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
52222025-04-30T06:43:52.765ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
52232025-04-30T06:43:52.767ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "071839ac-fc67-462e-97bc-5335e69559ed", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
52242025-04-30T06:43:52.767ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
52252025-04-30T06:43:52.774ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6a078e67-6796-4df5-9e26-baf58a686c74", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
52262025-04-30T06:43:52.774ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
52272025-04-30T06:43:52.776ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "42eea1ae-22a9-448a-be32-866cd33c22ab", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
52282025-04-30T06:43:52.776ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
52292025-04-30T06:43:52.780ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "95d4b0a2-3770-4fc4-88c0-39f1c3067242", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
52302025-04-30T06:43:52.780ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
52312025-04-30T06:43:52.786ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a386b761-c048-460a-a8e2-10337545e29c", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
52322025-04-30T06:43:52.786ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
52332025-04-30T06:43:52.789ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
52342025-04-30T06:43:52.792ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "36f81b8c-a955-41b4-864b-a77899c97049", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
52352025-04-30T06:43:52.792ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
52362025-04-30T06:43:52.795ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f26bd3b8-0e02-4dde-a568-a02878043982", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
52372025-04-30T06:43:52.795ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
52382025-04-30T06:43:52.797ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4f527036-4a40-4c4a-8905-64ea45d1f776", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
52392025-04-30T06:43:52.797ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
52402025-04-30T06:43:52.799ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "338e8fcd-250c-477b-86ea-f21a5505e761", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
52412025-04-30T06:43:52.799ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
52422025-04-30T06:43:52.800ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "26b90f07-e98d-4cdc-8413-fecc3055ba10", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
52432025-04-30T06:43:52.801ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
52442025-04-30T06:43:52.802ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3a4c1109-f5be-4933-939b-50b4b506ce9e", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
52452025-04-30T06:43:52.802ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
52462025-04-30T06:43:52.805ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e1122b40-3d22-4254-bb34-c1e07c40b40a", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
52472025-04-30T06:43:52.805ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
52482025-04-30T06:43:52.806ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e5419cc1-148d-4128-bc23-01d84db45083", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
52492025-04-30T06:43:52.807ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
52502025-04-30T06:43:52.807ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
52512025-04-30T06:43:52.807ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
52522025-04-30T06:43:52.807ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
52532025-04-30T06:43:52.808ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
52542025-04-30T06:43:52.811ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52552025-04-30T06:43:52.816ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
52562025-04-30T06:43:52.816ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
52572025-04-30T06:43:52.816ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
52582025-04-30T06:43:52.816ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
52592025-04-30T06:43:52.816ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
52602025-04-30T06:43:52.817ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
52612025-04-30T06:43:52.827ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52622025-04-30T06:43:52.836ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
52632025-04-30T06:43:52.837ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
52642025-04-30T06:43:52.837ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
52652025-04-30T06:43:52.837ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
52662025-04-30T06:43:52.840ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
52672025-04-30T06:43:52.840ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
52682025-04-30T06:43:52.840ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
52692025-04-30T06:43:52.840ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
52702025-04-30T06:43:52.842ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
52712025-04-30T06:43:52.847ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52722025-04-30T06:43:52.854ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
52732025-04-30T06:43:52.854ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
52742025-04-30T06:43:52.854ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
52752025-04-30T06:43:52.855ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
52762025-04-30T06:43:52.905ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
52772025-04-30T06:43:52.905ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory
52782025-04-30T06:43:52.905ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:32786 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
52792025-04-30T06:43:52.907ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
52802025-04-30T06:43:52.907ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
52812025-04-30T06:43:52.908ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
52822025-04-30T06:43:52.908ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:32786 req_id = 1a67cc3c-75db-4dc2-85fb-5910740f029e sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
52832025-04-30T06:43:52.909ZINFOtest_unauthorized (dropshot): request completed latency_us = 610 local_addr = [::1]:57786 method = GET remote_addr = [::1]:32786 req_id = 1a67cc3c-75db-4dc2-85fb-5910740f029e response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
52842025-04-30T06:43:52.909ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = http://[::1]:57786 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:57786/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a67cc3c-75db-4dc2-85fb-5910740f029e", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
52852025-04-30T06:43:52.911ZDEBGtest_unauthorized (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
52862025-04-30T06:43:52.911ZDEBGtest_unauthorized (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
52872025-04-30T06:43:52.911ZDEBGtest_unauthorized (BackgroundTasks): finished collection background_task = inventory_collection
52882025-04-30T06:43:52.912ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
52892025-04-30T06:43:52.913ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
52902025-04-30T06:43:52.913ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
52912025-04-30T06:43:52.913ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
52922025-04-30T06:43:52.914ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
52932025-04-30T06:43:52.920ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52942025-04-30T06:43:52.925ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
52952025-04-30T06:43:52.925ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
52962025-04-30T06:43:52.925ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
52972025-04-30T06:43:52.926ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
52982025-04-30T06:43:52.976ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
52992025-04-30T06:43:52.985ZWARNtest_unauthorized (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
53002025-04-30T06:43:52.985ZWARNtest_unauthorized (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
53012025-04-30T06:43:52.985ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 486.599006ms iteration = 13 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
53022025-04-30T06:43:52.985ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 14 reason = Signaled
53032025-04-30T06:43:52.985ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
53042025-04-30T06:43:52.985ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
53052025-04-30T06:43:52.985ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
53062025-04-30T06:43:52.985ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
53072025-04-30T06:43:52.986ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
53082025-04-30T06:43:52.990ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53092025-04-30T06:43:52.995ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
53102025-04-30T06:43:52.995ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
53112025-04-30T06:43:52.996ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
53122025-04-30T06:43:52.996ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
53132025-04-30T06:43:53.009ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1a8b2318-abf8-406d-a389-112ab91a0de8, 0)]
53142025-04-30T06:43:53.009ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
53152025-04-30T06:43:53.010ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: d3 d6 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:63806 req_id = 651f0922-f997-4096-a95f-032aa902173d
53162025-04-30T06:43:53.011ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 54230,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63806 req_id = 651f0922-f997-4096-a95f-032aa902173d
53172025-04-30T06:43:53.011ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
53182025-04-30T06:43:53.011ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
53192025-04-30T06:43:53.012ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:63806 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 651f0922-f997-4096-a95f-032aa902173d
53202025-04-30T06:43:53.012ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4297, tv_nsec: 811361004 } })
53212025-04-30T06:43:53.012ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
53222025-04-30T06:43:53.013ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 4a c1 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e J...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:51386 req_id = 66583d28-5cc8-424d-a904-65731da97aef
53232025-04-30T06:43:53.013ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 19137,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51386 req_id = 66583d28-5cc8-424d-a904-65731da97aef
53242025-04-30T06:43:53.013ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
53252025-04-30T06:43:53.013ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:51386 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 66583d28-5cc8-424d-a904-65731da97aef
53262025-04-30T06:43:53.082ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
53272025-04-30T06:43:53.082ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
53282025-04-30T06:43:53.083ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
53292025-04-30T06:43:53.083ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
53302025-04-30T06:43:53.083ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
53312025-04-30T06:43:53.084ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
53322025-04-30T06:43:53.084ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 78 93 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c x............_cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:62655 req_id = 7fc187e0-84c5-4ae3-93eb-ecdf91c5d837
53332025-04-30T06:43:53.084ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 30867,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62655 req_id = 7fc187e0-84c5-4ae3-93eb-ecdf91c5d837
53342025-04-30T06:43:53.085ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
53352025-04-30T06:43:53.085ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 30867, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:62655 req_id = 7fc187e0-84c5-4ae3-93eb-ecdf91c5d837
53362025-04-30T06:43:53.085ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
53372025-04-30T06:43:53.085ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
53382025-04-30T06:43:53.085ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
53392025-04-30T06:43:53.087ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53402025-04-30T06:43:53.093ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
53412025-04-30T06:43:53.094ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
53422025-04-30T06:43:53.094ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
53432025-04-30T06:43:53.098ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
53442025-04-30T06:43:53.100ZINFOtest_unauthorized (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
53452025-04-30T06:43:53.101ZDEBGtest_unauthorized (oximeter): client request body = None method = GET uri = http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
53462025-04-30T06:43:53.102ZTRACtest_unauthorized (dropshot_internal): incoming request local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 3bb64030-94f0-4a29-95fa-a36febd01ec2 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
53472025-04-30T06:43:53.105ZTRACtest_unauthorized (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 3bb64030-94f0-4a29-95fa-a36febd01ec2 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
53482025-04-30T06:43:53.105ZDEBGtest_unauthorized (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 3bb64030-94f0-4a29-95fa-a36febd01ec2 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
53492025-04-30T06:43:53.107ZDEBGtest_unauthorized (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 3bb64030-94f0-4a29-95fa-a36febd01ec2 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
53502025-04-30T06:43:53.108ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
53512025-04-30T06:43:53.108ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
53522025-04-30T06:43:53.111ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
53532025-04-30T06:43:53.119ZINFOtest_unauthorized (dropshot_internal): request completed latency_us = 17210 local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 3bb64030-94f0-4a29-95fa-a36febd01ec2 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
53542025-04-30T06:43:53.120ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "2adac445-5ee3-4761-9605-2e65d7f64946", "content-length": "835", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
53552025-04-30T06:43:53.125ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
53562025-04-30T06:43:53.126ZDEBGtest_unauthorized (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "3bb64030-94f0-4a29-95fa-a36febd01ec2", "content-length": "405", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
53572025-04-30T06:43:53.126ZDEBGtest_unauthorized (oximeter): client request body = None method = GET uri = http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
53582025-04-30T06:43:53.127ZTRACtest_unauthorized (dropshot_internal): incoming request local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 915512c9-809f-4817-bbb4-68ae2a9bec4f uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
53592025-04-30T06:43:53.127ZTRACtest_unauthorized (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 915512c9-809f-4817-bbb4-68ae2a9bec4f resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
53602025-04-30T06:43:53.127ZDEBGtest_unauthorized (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 915512c9-809f-4817-bbb4-68ae2a9bec4f roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
53612025-04-30T06:43:53.128ZDEBGtest_unauthorized (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 915512c9-809f-4817-bbb4-68ae2a9bec4f resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
53622025-04-30T06:43:53.132ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "17da2b7b-25e0-478f-89d1-703ea8b38b25", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
53632025-04-30T06:43:53.132ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
53642025-04-30T06:43:53.134ZINFOtest_unauthorized (dropshot_internal): request completed latency_us = 6152 local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 915512c9-809f-4817-bbb4-68ae2a9bec4f response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
53652025-04-30T06:43:53.136ZDEBGtest_unauthorized (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "915512c9-809f-4817-bbb4-68ae2a9bec4f", "content-length": "29", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
53662025-04-30T06:43:53.136ZTRACtest_unauthorized (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 8d752adb-08b9-405f-a11b-66019bf447b5
53672025-04-30T06:43:53.136ZTRACtest_unauthorized (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
53682025-04-30T06:43:53.136ZINFOtest_unauthorized (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
53692025-04-30T06:43:53.140ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "87fa3909-8687-4357-b471-29c39c118b36", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
53702025-04-30T06:43:53.151ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
53712025-04-30T06:43:53.154ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "611b579b-1020-44c2-9ee5-6eac50c9ac1a", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
53722025-04-30T06:43:53.154ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
53732025-04-30T06:43:53.156ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9a3d50bf-5f8c-42d3-9b28-512a0c2bc680", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
53742025-04-30T06:43:53.157ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
53752025-04-30T06:43:53.161ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8189d6f9-ffba-45b5-85b9-4e0348b6420f", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
53762025-04-30T06:43:53.162ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
53772025-04-30T06:43:53.164ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3b9fad4f-15b9-4a35-bbb8-693e6f251876", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
53782025-04-30T06:43:53.164ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
53792025-04-30T06:43:53.170ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
53802025-04-30T06:43:53.172ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "98010c2c-28dd-43f5-8d86-543d90531459", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
53812025-04-30T06:43:53.172ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
53822025-04-30T06:43:53.173ZDEBGtest_unauthorized (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
53832025-04-30T06:43:53.173ZDEBGtest_unauthorized (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
53842025-04-30T06:43:53.174ZDEBGtest_unauthorized (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a
53852025-04-30T06:43:53.176ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
53862025-04-30T06:43:53.176ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6bb15755-a1e9-4ce6-9841-3eeae29802ee", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
53872025-04-30T06:43:53.176ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
53882025-04-30T06:43:53.178ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b53e753b-ab3c-4883-b20e-b9e88443ea79", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
53892025-04-30T06:43:53.178ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
53902025-04-30T06:43:53.179ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9b240e26-00ca-40e9-9b76-e3280bbfebf9", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
53912025-04-30T06:43:53.179ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
53922025-04-30T06:43:53.180ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 2 table_name = fields_string
53932025-04-30T06:43:53.180ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
53942025-04-30T06:43:53.181ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f26c50a6-a529-4042-bf3f-5423396c3cf4", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
53952025-04-30T06:43:53.181ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
53962025-04-30T06:43:53.183ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "63d03d61-d1ff-4ff6-99b5-2a8e045ab960", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
53972025-04-30T06:43:53.183ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
53982025-04-30T06:43:53.183ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 1 table_name = measurements_i64
53992025-04-30T06:43:53.183ZTRACtest_unauthorized (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
54002025-04-30T06:43:53.185ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8df487ce-f7c5-437e-a215-f9a38a747aa3", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
54012025-04-30T06:43:53.185ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
54022025-04-30T06:43:53.188ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a776496e-bbcb-470b-8290-2fbf11c91fca", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
54032025-04-30T06:43:53.188ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
54042025-04-30T06:43:53.191ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "545ef17d-fcd3-4f91-9984-409deb5206d7", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
54052025-04-30T06:43:53.191ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
54062025-04-30T06:43:53.193ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "72eeb193-5fcd-43d2-89a2-232822c891be", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
54072025-04-30T06:43:53.193ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
54082025-04-30T06:43:53.200ZDEBGtest_unauthorized (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
54092025-04-30T06:43:53.200ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a170bb33-6028-4b66-8859-0825ea2a26b8", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
54102025-04-30T06:43:53.201ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
54112025-04-30T06:43:53.201ZDEBGtest_unauthorized (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
54122025-04-30T06:43:53.207ZTRACtest_unauthorized (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
54132025-04-30T06:43:53.212ZDEBGtest_unauthorized (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
54142025-04-30T06:43:53.212ZTRACtest_unauthorized (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
54152025-04-30T06:43:53.212ZTRACtest_unauthorized (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
54162025-04-30T06:43:53.215ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c3460d6f-6565-4b51-98d6-3bba898cf1f7", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
54172025-04-30T06:43:53.215ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
54182025-04-30T06:43:53.216ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "60d1b297-0ead-429e-a283-ff9d83558678", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
54192025-04-30T06:43:53.217ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
54202025-04-30T06:43:53.217ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e7d73bb6-3007-4053-8d6d-c40bfe0a5b7f", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
54212025-04-30T06:43:53.218ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
54222025-04-30T06:43:53.219ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "71fd0246-e20a-4df7-bae0-559aeacd2b48", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
54232025-04-30T06:43:53.219ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
54242025-04-30T06:43:53.221ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
54252025-04-30T06:43:53.221ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
54262025-04-30T06:43:53.221ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
54272025-04-30T06:43:53.221ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
54282025-04-30T06:43:53.222ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
54292025-04-30T06:43:53.222ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a06fa9c-436f-40d8-baa3-2784bdbea521", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
54302025-04-30T06:43:53.225ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
54312025-04-30T06:43:53.225ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "970c3864-0584-4a70-882a-bb3f4c27acfb", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
54322025-04-30T06:43:53.225ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
54332025-04-30T06:43:53.226ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "584130f2-33e9-426c-8e88-fa8aa48993c5", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
54342025-04-30T06:43:53.226ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
54352025-04-30T06:43:53.228ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "66ccf310-5bb9-492e-8054-2185f272b559", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
54362025-04-30T06:43:53.228ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
54372025-04-30T06:43:53.232ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54382025-04-30T06:43:53.239ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
54392025-04-30T06:43:53.239ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
54402025-04-30T06:43:53.239ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
54412025-04-30T06:43:53.240ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
54422025-04-30T06:43:53.240ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0a68dcaa-3bbf-4e51-a43d-74bd8b81d8cb", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
54432025-04-30T06:43:53.240ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
54442025-04-30T06:43:53.244ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
54452025-04-30T06:43:53.244ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
54462025-04-30T06:43:53.244ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
54472025-04-30T06:43:53.245ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b41c858a-2e99-4b48-af96-56194392a13d", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
54482025-04-30T06:43:53.245ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
54492025-04-30T06:43:53.248ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "41699e15-a05b-4930-b3cf-8f486f18232d", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
54502025-04-30T06:43:53.248ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
54512025-04-30T06:43:53.252ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "284e02f1-f00c-4600-b646-7ccd760c3ade", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
54522025-04-30T06:43:53.252ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
54532025-04-30T06:43:53.253ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "754d195e-efaa-466f-96e9-b11a76c6d628", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
54542025-04-30T06:43:53.253ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
54552025-04-30T06:43:53.254ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0ec376e-2cf8-4411-a679-d87a7006f451", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
54562025-04-30T06:43:53.254ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
54572025-04-30T06:43:53.255ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "99a7de62-01d0-42c9-84f4-4cea72406cf4", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
54582025-04-30T06:43:53.255ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
54592025-04-30T06:43:53.258ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0c7ce589-1ed5-4115-9684-cc382e70ef61", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
54602025-04-30T06:43:53.258ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
54612025-04-30T06:43:53.260ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9b86da9a-f791-4d41-b544-893ceee69f05", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
54622025-04-30T06:43:53.260ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
54632025-04-30T06:43:53.263ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "db4028a6-3ef8-4202-9b82-e2adbf61333e", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
54642025-04-30T06:43:53.263ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
54652025-04-30T06:43:53.268ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "893f0f38-ea36-4fc8-9091-c1cc650f5161", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
54662025-04-30T06:43:53.268ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
54672025-04-30T06:43:53.272ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "12169bc9-0fd7-43c9-bb0b-46b4b24566b4", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:52 GMT"} })
54682025-04-30T06:43:53.272ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
54692025-04-30T06:43:53.275ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
54702025-04-30T06:43:53.276ZWARNtest_unauthorized (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
54712025-04-30T06:43:53.278ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9c8cc185-284a-4649-8cc4-1dd6e3d977fa", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
54722025-04-30T06:43:53.278ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
54732025-04-30T06:43:53.281ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "67e2b0f2-26f0-4809-ab73-791181f1e11e", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
54742025-04-30T06:43:53.281ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
54752025-04-30T06:43:53.283ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "04cdc250-2260-40ce-af47-3ce1fc9d118e", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
54762025-04-30T06:43:53.283ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
54772025-04-30T06:43:53.283ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3eeb5606-ea13-4f54-8976-4934903096dc", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
54782025-04-30T06:43:53.284ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
54792025-04-30T06:43:53.284ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f352d1bb-5d5b-43aa-bf3d-c22a6ee95eda", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
54802025-04-30T06:43:53.285ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
54812025-04-30T06:43:53.288ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0a4b0cde-7598-4dff-9f19-d32a7bdf4bf6", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
54822025-04-30T06:43:53.288ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
54832025-04-30T06:43:53.289ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5fc18eaf-5bc5-4b93-a605-4eb4a4cd6f00", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
54842025-04-30T06:43:53.289ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
54852025-04-30T06:43:53.289ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
54862025-04-30T06:43:53.289ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
54872025-04-30T06:43:53.289ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
54882025-04-30T06:43:53.290ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
54892025-04-30T06:43:53.296ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54902025-04-30T06:43:53.300ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
54912025-04-30T06:43:53.300ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
54922025-04-30T06:43:53.300ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
54932025-04-30T06:43:53.300ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
54942025-04-30T06:43:53.300ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
54952025-04-30T06:43:53.301ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
54962025-04-30T06:43:53.304ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54972025-04-30T06:43:53.308ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
54982025-04-30T06:43:53.308ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
54992025-04-30T06:43:53.308ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
55002025-04-30T06:43:53.308ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
55012025-04-30T06:43:53.367ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
55022025-04-30T06:43:53.367ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory
55032025-04-30T06:43:53.367ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
55042025-04-30T06:43:53.367ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
55052025-04-30T06:43:53.367ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
55062025-04-30T06:43:53.367ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
55072025-04-30T06:43:53.368ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
55082025-04-30T06:43:53.368ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:38812 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
55092025-04-30T06:43:53.369ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:38812 req_id = 5a52991b-f11d-4ba2-8831-42af86616bb2 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
55102025-04-30T06:43:53.370ZINFOtest_unauthorized (dropshot): request completed latency_us = 435 local_addr = [::1]:57786 method = GET remote_addr = [::1]:38812 req_id = 5a52991b-f11d-4ba2-8831-42af86616bb2 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
55112025-04-30T06:43:53.372ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55122025-04-30T06:43:53.378ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
55132025-04-30T06:43:53.378ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
55142025-04-30T06:43:53.378ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
55152025-04-30T06:43:53.378ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
55162025-04-30T06:43:53.379ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = http://[::1]:57786 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:57786/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "5a52991b-f11d-4ba2-8831-42af86616bb2", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
55172025-04-30T06:43:53.381ZDEBGtest_unauthorized (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
55182025-04-30T06:43:53.381ZDEBGtest_unauthorized (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
55192025-04-30T06:43:53.381ZDEBGtest_unauthorized (BackgroundTasks): finished collection background_task = inventory_collection
55202025-04-30T06:43:53.383ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
55212025-04-30T06:43:53.383ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
55222025-04-30T06:43:53.383ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
55232025-04-30T06:43:53.383ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
55242025-04-30T06:43:53.384ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
55252025-04-30T06:43:53.387ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
55262025-04-30T06:43:53.387ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
55272025-04-30T06:43:53.388ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
55282025-04-30T06:43:53.388ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55292025-04-30T06:43:53.392ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
55302025-04-30T06:43:53.392ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
55312025-04-30T06:43:53.392ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
55322025-04-30T06:43:53.392ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
55332025-04-30T06:43:53.426ZWARNtest_unauthorized (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
55342025-04-30T06:43:53.426ZWARNtest_unauthorized (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
55352025-04-30T06:43:53.427ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 442.032301ms iteration = 14 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
55362025-04-30T06:43:53.427ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 15 reason = Signaled
55372025-04-30T06:43:53.428ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
55382025-04-30T06:43:53.428ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
55392025-04-30T06:43:53.428ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
55402025-04-30T06:43:53.428ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
55412025-04-30T06:43:53.429ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
55422025-04-30T06:43:53.433ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55432025-04-30T06:43:53.439ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
55442025-04-30T06:43:53.439ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
55452025-04-30T06:43:53.439ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
55462025-04-30T06:43:53.440ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
55472025-04-30T06:43:53.445ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
55482025-04-30T06:43:53.446ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1a8b2318-abf8-406d-a389-112ab91a0de8, 0)]
55492025-04-30T06:43:53.447ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
55502025-04-30T06:43:53.448ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: c7 29 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .)..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:54585 req_id = 3671d0de-91aa-4e28-8309-ef5add6d0c90
55512025-04-30T06:43:53.448ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 50985,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54585 req_id = 3671d0de-91aa-4e28-8309-ef5add6d0c90
55522025-04-30T06:43:53.449ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
55532025-04-30T06:43:53.449ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
55542025-04-30T06:43:53.450ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:54585 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 3671d0de-91aa-4e28-8309-ef5add6d0c90
55552025-04-30T06:43:53.450ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4298, tv_nsec: 248465787 } })
55562025-04-30T06:43:53.450ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
55572025-04-30T06:43:53.450ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 98 a6 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:62674 req_id = c9c3444e-7713-4393-884e-e65bee22d2d0
55582025-04-30T06:43:53.450ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 39078,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62674 req_id = c9c3444e-7713-4393-884e-e65bee22d2d0
55592025-04-30T06:43:53.450ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
55602025-04-30T06:43:53.450ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:62674 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = c9c3444e-7713-4393-884e-e65bee22d2d0
55612025-04-30T06:43:53.522ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
55622025-04-30T06:43:53.523ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
55632025-04-30T06:43:53.523ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
55642025-04-30T06:43:53.523ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
55652025-04-30T06:43:53.523ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
55662025-04-30T06:43:53.524ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
55672025-04-30T06:43:53.525ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 9b d1 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:62449 req_id = 365eb380-ed30-4961-b278-cb5a762c2257
55682025-04-30T06:43:53.525ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 39889,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62449 req_id = 365eb380-ed30-4961-b278-cb5a762c2257
55692025-04-30T06:43:53.525ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
55702025-04-30T06:43:53.525ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 39889, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:62449 req_id = 365eb380-ed30-4961-b278-cb5a762c2257
55712025-04-30T06:43:53.526ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
55722025-04-30T06:43:53.526ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
55732025-04-30T06:43:53.527ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
55742025-04-30T06:43:53.528ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55752025-04-30T06:43:53.543ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
55762025-04-30T06:43:53.543ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
55772025-04-30T06:43:53.544ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
55782025-04-30T06:43:53.544ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
55792025-04-30T06:43:53.567ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "d9e25da8-1028-4a26-a194-70002270aac2", "content-length": "835", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
55802025-04-30T06:43:53.569ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
55812025-04-30T06:43:53.569ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
55822025-04-30T06:43:53.569ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
55832025-04-30T06:43:53.570ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
55842025-04-30T06:43:53.593ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6fe8eb08-1dc2-433c-87de-d81962132ec0", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
55852025-04-30T06:43:53.593ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
55862025-04-30T06:43:53.633ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7b0459f1-256b-4737-b6cf-c9ba61968ba2", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
55872025-04-30T06:43:53.633ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
55882025-04-30T06:43:53.638ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "608d1ed6-26ea-43c7-8c21-16485a55fe2e", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
55892025-04-30T06:43:53.638ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
55902025-04-30T06:43:53.642ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "738be779-1e8a-4b93-899d-9bbba92bbe70", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
55912025-04-30T06:43:53.649ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
55922025-04-30T06:43:53.655ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
55932025-04-30T06:43:53.661ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6f500230-d340-4ed2-96c2-17ae83a63e21", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
55942025-04-30T06:43:53.661ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
55952025-04-30T06:43:53.665ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "13e6b115-d44a-41b3-99d8-a47ce831dd7c", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
55962025-04-30T06:43:53.666ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
55972025-04-30T06:43:53.668ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "057a517b-7b64-453d-82d5-f9f4d5b14165", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
55982025-04-30T06:43:53.669ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
55992025-04-30T06:43:53.670ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a5ff9d97-1835-44bc-a8de-5965d266af53", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
56002025-04-30T06:43:53.670ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
56012025-04-30T06:43:53.671ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d84f8d33-7a34-487e-8267-c088c5dabe37", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
56022025-04-30T06:43:53.671ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
56032025-04-30T06:43:53.672ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "dce160f1-002a-42de-82cf-f0f6d5843137", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
56042025-04-30T06:43:53.672ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
56052025-04-30T06:43:53.673ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "774a4525-ec77-4135-8cd2-6a8c91a7a681", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
56062025-04-30T06:43:53.673ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
56072025-04-30T06:43:53.676ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "475cec82-929c-4eae-b30e-ecb45292b6c0", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
56082025-04-30T06:43:53.677ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
56092025-04-30T06:43:53.680ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d36c6c69-e6db-4b6b-995f-8c7075b5da44", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
56102025-04-30T06:43:53.680ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
56112025-04-30T06:43:53.683ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e03294ff-d676-4547-bfeb-2702ad235beb", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
56122025-04-30T06:43:53.683ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
56132025-04-30T06:43:53.685ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "210e5bc5-b427-488b-b2c9-7b6b5ce11062", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
56142025-04-30T06:43:53.685ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
56152025-04-30T06:43:53.688ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ab376028-df6c-49f5-b202-fdb03a701108", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
56162025-04-30T06:43:53.689ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
56172025-04-30T06:43:53.693ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "52f5f531-e92f-4d4c-b76b-63b0d07ef25f", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
56182025-04-30T06:43:53.693ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
56192025-04-30T06:43:53.700ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2a72ff3e-dcf1-4a0c-bd84-a91941460a5c", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
56202025-04-30T06:43:53.700ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
56212025-04-30T06:43:53.704ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "021fc3da-11ae-4ace-a164-68c4f3463507", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
56222025-04-30T06:43:53.704ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
56232025-04-30T06:43:53.706ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ee991324-4bd7-423f-b218-18f9bb545a4b", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
56242025-04-30T06:43:53.706ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
56252025-04-30T06:43:53.707ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
56262025-04-30T06:43:53.707ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
56272025-04-30T06:43:53.707ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
56282025-04-30T06:43:53.707ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
56292025-04-30T06:43:53.708ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
56302025-04-30T06:43:53.709ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6e1a9668-2d40-4d8c-881a-b48cee73ad96", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
56312025-04-30T06:43:53.709ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
56322025-04-30T06:43:53.711ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56332025-04-30T06:43:53.718ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
56342025-04-30T06:43:53.718ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
56352025-04-30T06:43:53.718ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
56362025-04-30T06:43:53.719ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
56372025-04-30T06:43:53.720ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ef3c927b-a952-4390-a54b-2a976d515e27", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
56382025-04-30T06:43:53.720ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
56392025-04-30T06:43:53.721ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
56402025-04-30T06:43:53.721ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
56412025-04-30T06:43:53.722ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
56422025-04-30T06:43:53.724ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "87590e1d-0bf3-4564-b659-13a2de47f390", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
56432025-04-30T06:43:53.724ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
56442025-04-30T06:43:53.726ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1addcd48-04d5-430b-b1b5-43dc2edad72f", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
56452025-04-30T06:43:53.727ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
56462025-04-30T06:43:53.729ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e7162fb5-e546-4ca5-9cb5-d217d1137531", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
56472025-04-30T06:43:53.729ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
56482025-04-30T06:43:53.734ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f600ccba-c001-4c2b-885d-d505a6abe370", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
56492025-04-30T06:43:53.734ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
56502025-04-30T06:43:53.741ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6a8a5e81-90b3-41cd-be67-24f31331d7eb", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
56512025-04-30T06:43:53.741ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
56522025-04-30T06:43:53.743ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3412d13d-9ac9-4bde-b609-6ae658fe51ee", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
56532025-04-30T06:43:53.744ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
56542025-04-30T06:43:53.746ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "83b3a550-9e43-4cbb-84c9-78539a33cff3", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
56552025-04-30T06:43:53.746ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
56562025-04-30T06:43:53.748ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3db0b61c-02fd-4db9-b08c-305f0d08316b", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
56572025-04-30T06:43:53.749ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
56582025-04-30T06:43:53.753ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
56592025-04-30T06:43:53.755ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8d8e2f1e-58fb-4c18-8b89-1db9c8639b40", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
56602025-04-30T06:43:53.755ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
56612025-04-30T06:43:53.757ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "323f7588-0182-44c1-82a4-7551bb730e34", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
56622025-04-30T06:43:53.757ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
56632025-04-30T06:43:53.760ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d34b87d4-d592-4371-b412-8680f3b35077", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
56642025-04-30T06:43:53.760ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
56652025-04-30T06:43:53.761ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5d19f664-fb12-4ba9-a609-bd431f488ac0", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
56662025-04-30T06:43:53.761ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
56672025-04-30T06:43:53.764ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1cb4f5d6-8171-4ebe-b14f-5c6e10c9d21f", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
56682025-04-30T06:43:53.764ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
56692025-04-30T06:43:53.766ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e8cc9ab8-d7f7-4d02-800a-e838c057b18a", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
56702025-04-30T06:43:53.766ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
56712025-04-30T06:43:53.768ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "54e434e1-450f-4bc7-b4a7-0b117f906202", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
56722025-04-30T06:43:53.768ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
56732025-04-30T06:43:53.771ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "16db093a-c282-49d7-89cf-52138d55df2d", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
56742025-04-30T06:43:53.771ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
56752025-04-30T06:43:53.774ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "047ece53-f2e5-49ac-ab24-40801d540c17", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
56762025-04-30T06:43:53.774ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
56772025-04-30T06:43:53.779ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "95826b01-276a-4e54-95c2-20e53c14e8f2", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
56782025-04-30T06:43:53.779ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
56792025-04-30T06:43:53.786ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5f6f1aaf-ff85-4042-9a0d-d32c3c82c6b2", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
56802025-04-30T06:43:53.786ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
56812025-04-30T06:43:53.788ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3bdbb7e0-8e4e-423d-a81f-53795e363ab2", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
56822025-04-30T06:43:53.788ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
56832025-04-30T06:43:53.789ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f704a47f-c1a4-4c06-ade3-da4544ead662", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
56842025-04-30T06:43:53.789ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
56852025-04-30T06:43:53.790ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b0a393e1-b64b-49c9-84e7-20c6dbb3fd07", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
56862025-04-30T06:43:53.790ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
56872025-04-30T06:43:53.790ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
56882025-04-30T06:43:53.790ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
56892025-04-30T06:43:53.790ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
56902025-04-30T06:43:53.792ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
56912025-04-30T06:43:53.795ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56922025-04-30T06:43:53.803ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
56932025-04-30T06:43:53.803ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
56942025-04-30T06:43:53.804ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
56952025-04-30T06:43:53.804ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
56962025-04-30T06:43:53.804ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
56972025-04-30T06:43:53.804ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
56982025-04-30T06:43:53.805ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
56992025-04-30T06:43:53.805ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
57002025-04-30T06:43:53.806ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
57012025-04-30T06:43:53.806ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
57022025-04-30T06:43:53.807ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
57032025-04-30T06:43:53.809ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57042025-04-30T06:43:53.814ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
57052025-04-30T06:43:53.814ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
57062025-04-30T06:43:53.814ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
57072025-04-30T06:43:53.815ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
57082025-04-30T06:43:53.819ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57092025-04-30T06:43:53.830ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
57102025-04-30T06:43:53.831ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
57112025-04-30T06:43:53.831ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
57122025-04-30T06:43:53.831ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
57132025-04-30T06:43:53.906ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
57142025-04-30T06:43:53.906ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory
57152025-04-30T06:43:53.926ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:57136 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
57162025-04-30T06:43:53.927ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:57136 req_id = 44aaa06c-ce61-4225-922f-53bf00f69be6 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
57172025-04-30T06:43:53.927ZINFOtest_unauthorized (dropshot): request completed latency_us = 486 local_addr = [::1]:57786 method = GET remote_addr = [::1]:57136 req_id = 44aaa06c-ce61-4225-922f-53bf00f69be6 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
57182025-04-30T06:43:53.927ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
57192025-04-30T06:43:53.927ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
57202025-04-30T06:43:53.928ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
57212025-04-30T06:43:53.928ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = http://[::1]:57786 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:57786/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "44aaa06c-ce61-4225-922f-53bf00f69be6", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
57222025-04-30T06:43:53.931ZDEBGtest_unauthorized (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
57232025-04-30T06:43:53.931ZDEBGtest_unauthorized (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
57242025-04-30T06:43:53.931ZDEBGtest_unauthorized (BackgroundTasks): finished collection background_task = inventory_collection
57252025-04-30T06:43:53.936ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
57262025-04-30T06:43:53.936ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
57272025-04-30T06:43:53.936ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
57282025-04-30T06:43:53.936ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
57292025-04-30T06:43:53.938ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
57302025-04-30T06:43:53.955ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57312025-04-30T06:43:53.962ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
57322025-04-30T06:43:53.962ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
57332025-04-30T06:43:53.962ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
57342025-04-30T06:43:53.964ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
57352025-04-30T06:43:54.056ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
57362025-04-30T06:43:54.057ZWARNtest_unauthorized (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
57372025-04-30T06:43:54.058ZWARNtest_unauthorized (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
57382025-04-30T06:43:54.058ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 630.684357ms iteration = 15 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
57392025-04-30T06:43:54.058ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 16 reason = Signaled
57402025-04-30T06:43:54.058ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
57412025-04-30T06:43:54.058ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
57422025-04-30T06:43:54.058ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
57432025-04-30T06:43:54.058ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
57442025-04-30T06:43:54.059ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
57452025-04-30T06:43:54.061ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57462025-04-30T06:43:54.066ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
57472025-04-30T06:43:54.066ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
57482025-04-30T06:43:54.066ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
57492025-04-30T06:43:54.066ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
57502025-04-30T06:43:54.080ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1a8b2318-abf8-406d-a389-112ab91a0de8, 0)]
57512025-04-30T06:43:54.080ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
57522025-04-30T06:43:54.081ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 72 f2 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 r............_mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:64778 req_id = c85c107c-a72f-4095-baec-68c5e1f5ca6f
57532025-04-30T06:43:54.081ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 29426,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64778 req_id = c85c107c-a72f-4095-baec-68c5e1f5ca6f
57542025-04-30T06:43:54.082ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
57552025-04-30T06:43:54.088ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
57562025-04-30T06:43:54.088ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:64778 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = c85c107c-a72f-4095-baec-68c5e1f5ca6f
57572025-04-30T06:43:54.091ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4298, tv_nsec: 890712401 } })
57582025-04-30T06:43:54.092ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
57592025-04-30T06:43:54.093ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 85 58 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .X..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:65097 req_id = ce44b0e2-e68f-4d3a-8a90-1e4fe739b513
57602025-04-30T06:43:54.093ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 34136,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:65097 req_id = ce44b0e2-e68f-4d3a-8a90-1e4fe739b513
57612025-04-30T06:43:54.093ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
57622025-04-30T06:43:54.093ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:65097 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = ce44b0e2-e68f-4d3a-8a90-1e4fe739b513
57632025-04-30T06:43:54.185ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
57642025-04-30T06:43:54.186ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
57652025-04-30T06:43:54.186ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
57662025-04-30T06:43:54.186ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
57672025-04-30T06:43:54.186ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
57682025-04-30T06:43:54.188ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
57692025-04-30T06:43:54.188ZDEBGtest_unauthorized (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
57702025-04-30T06:43:54.188ZDEBGtest_unauthorized (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
57712025-04-30T06:43:54.189ZDEBGtest_unauthorized (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a
57722025-04-30T06:43:54.190ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
57732025-04-30T06:43:54.196ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 8f 59 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c .Y..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:56736 req_id = adb59431-b9fb-4c55-b1da-d2a9f7bfdf5f
57742025-04-30T06:43:54.196ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 36697,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:56736 req_id = adb59431-b9fb-4c55-b1da-d2a9f7bfdf5f
57752025-04-30T06:43:54.196ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
57762025-04-30T06:43:54.196ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 36697, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:56736 req_id = adb59431-b9fb-4c55-b1da-d2a9f7bfdf5f
57772025-04-30T06:43:54.197ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57782025-04-30T06:43:54.201ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
57792025-04-30T06:43:54.201ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
57802025-04-30T06:43:54.202ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
57812025-04-30T06:43:54.202ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
57822025-04-30T06:43:54.202ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 2 table_name = fields_string
57832025-04-30T06:43:54.203ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
57842025-04-30T06:43:54.204ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
57852025-04-30T06:43:54.204ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
57862025-04-30T06:43:54.204ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
57872025-04-30T06:43:54.208ZDEBGtest_unauthorized (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
57882025-04-30T06:43:54.208ZDEBGtest_unauthorized (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
57892025-04-30T06:43:54.209ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 1 table_name = measurements_i64
57902025-04-30T06:43:54.209ZTRACtest_unauthorized (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
57912025-04-30T06:43:54.211ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
57922025-04-30T06:43:54.211ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
57932025-04-30T06:43:54.211ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
57942025-04-30T06:43:54.212ZTRACtest_unauthorized (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
57952025-04-30T06:43:54.213ZDEBGtest_unauthorized (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
57962025-04-30T06:43:54.213ZTRACtest_unauthorized (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
57972025-04-30T06:43:54.213ZTRACtest_unauthorized (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
57982025-04-30T06:43:54.215ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "ebc210bc-8978-48fe-9c22-d323ed26db93", "content-length": "835", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
57992025-04-30T06:43:54.215ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
58002025-04-30T06:43:54.217ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "986c6dce-94e2-4a1c-babd-9224824349f5", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
58012025-04-30T06:43:54.224ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
58022025-04-30T06:43:54.226ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d089e395-7251-44dc-a1ad-8e109bef714e", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
58032025-04-30T06:43:54.227ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
58042025-04-30T06:43:54.241ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "67af61e7-abc9-400e-8d53-3a749edbe7ba", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
58052025-04-30T06:43:54.241ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
58062025-04-30T06:43:54.242ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "56694740-dbce-40e9-870c-9ed3e7592519", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
58072025-04-30T06:43:54.242ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
58082025-04-30T06:43:54.245ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a96ca5af-666b-4866-9dce-b5b75a2316d2", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
58092025-04-30T06:43:54.245ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
58102025-04-30T06:43:54.247ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "758a7282-c8af-4275-8238-30d6df21e6ce", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
58112025-04-30T06:43:54.247ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
58122025-04-30T06:43:54.250ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d2c64169-a76f-47c9-9ff5-36553cb825db", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
58132025-04-30T06:43:54.250ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
58142025-04-30T06:43:54.256ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "215728d0-27c9-40d6-b1c7-1120bd28f638", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
58152025-04-30T06:43:54.256ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
58162025-04-30T06:43:54.257ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5147da39-feb1-40c5-aae0-ef06d0a3a9d3", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
58172025-04-30T06:43:54.261ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
58182025-04-30T06:43:54.261ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "10993e2c-0f56-4621-8002-ab7011d6a1dc", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
58192025-04-30T06:43:54.261ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
58202025-04-30T06:43:54.262ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b1fd5915-60e2-455f-baf0-6ad58de462f6", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
58212025-04-30T06:43:54.262ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
58222025-04-30T06:43:54.263ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4435658e-85f6-4544-88db-0b9f5eeecddc", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
58232025-04-30T06:43:54.264ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
58242025-04-30T06:43:54.267ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
58252025-04-30T06:43:54.268ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ce249b9a-2a58-44a6-92e2-268bdd2892c0", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
58262025-04-30T06:43:54.269ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
58272025-04-30T06:43:54.271ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e0af706f-5622-4a2a-8e1d-224fa19e29f5", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
58282025-04-30T06:43:54.271ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
58292025-04-30T06:43:54.273ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "336714b3-dc51-4a9e-b19a-1e2f5ca58e36", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
58302025-04-30T06:43:54.273ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
58312025-04-30T06:43:54.275ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "066afe50-8686-4374-a4bb-0ae39a50404b", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
58322025-04-30T06:43:54.275ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
58332025-04-30T06:43:54.277ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0caf239-628f-4170-b058-a6e31d75a0e9", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
58342025-04-30T06:43:54.277ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
58352025-04-30T06:43:54.279ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "06a68e95-21a7-432b-bdf3-9328c7424b11", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
58362025-04-30T06:43:54.279ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
58372025-04-30T06:43:54.280ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc9d26c9-e504-4294-b2e9-8f0d4726d56a", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
58382025-04-30T06:43:54.280ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
58392025-04-30T06:43:54.281ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "112d361f-7105-4ae3-a43e-e130cb433b82", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
58402025-04-30T06:43:54.282ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
58412025-04-30T06:43:54.285ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "03b1bac4-b8b9-4784-a49c-54b03aab1617", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
58422025-04-30T06:43:54.285ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
58432025-04-30T06:43:54.287ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8f3c5bef-5743-4f8a-9299-c486ddc7f9d0", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
58442025-04-30T06:43:54.287ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
58452025-04-30T06:43:54.289ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ec9cf5f6-ab7b-4cf0-89b9-d13aa500b164", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
58462025-04-30T06:43:54.289ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
58472025-04-30T06:43:54.290ZWARNtest_unauthorized (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
58482025-04-30T06:43:54.291ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "54d50c40-3c73-4fe7-888d-339cb157b49a", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
58492025-04-30T06:43:54.291ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
58502025-04-30T06:43:54.293ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1ead14d5-b09d-451e-8a32-536f78e48e81", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
58512025-04-30T06:43:54.307ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
58522025-04-30T06:43:54.310ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "90c78e84-cba6-464d-aee8-4b5e98f9ee04", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
58532025-04-30T06:43:54.310ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
58542025-04-30T06:43:54.323ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
58552025-04-30T06:43:54.323ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
58562025-04-30T06:43:54.323ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
58572025-04-30T06:43:54.323ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
58582025-04-30T06:43:54.324ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
58592025-04-30T06:43:54.326ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "db72e3ce-adec-430f-a2ed-3c65244ebfa9", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
58602025-04-30T06:43:54.326ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
58612025-04-30T06:43:54.327ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58622025-04-30T06:43:54.347ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
58632025-04-30T06:43:54.347ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
58642025-04-30T06:43:54.348ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
58652025-04-30T06:43:54.348ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
58662025-04-30T06:43:54.375ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
58672025-04-30T06:43:54.375ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
58682025-04-30T06:43:54.376ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
58692025-04-30T06:43:54.376ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "093f5322-99e2-4ca6-9be9-1d6499711976", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
58702025-04-30T06:43:54.376ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
58712025-04-30T06:43:54.381ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5ebb140b-4c26-4cc6-a659-edc8406753ca", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
58722025-04-30T06:43:54.381ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
58732025-04-30T06:43:54.383ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "89274262-c737-4371-89a1-d899eb07e473", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
58742025-04-30T06:43:54.383ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
58752025-04-30T06:43:54.385ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ef69b07d-f246-4674-9a12-d619efccbb81", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
58762025-04-30T06:43:54.385ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
58772025-04-30T06:43:54.387ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e4ca48d4-04da-409b-a821-a15691bb5a49", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
58782025-04-30T06:43:54.387ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
58792025-04-30T06:43:54.391ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7b979f24-4f4b-458e-8c05-52c1c9bc1d71", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
58802025-04-30T06:43:54.391ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
58812025-04-30T06:43:54.392ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b7ea1301-343c-4840-88a8-2ce785acad47", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
58822025-04-30T06:43:54.393ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
58832025-04-30T06:43:54.396ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6b5ea07a-88bd-436b-b568-409ca4b312aa", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
58842025-04-30T06:43:54.396ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
58852025-04-30T06:43:54.419ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e89f8fb3-8560-4000-b21c-a4c163fb48d1", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
58862025-04-30T06:43:54.419ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
58872025-04-30T06:43:54.424ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "43042258-2946-41b0-bb6a-92c809448d1a", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
58882025-04-30T06:43:54.424ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
58892025-04-30T06:43:54.430ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "88c8d217-9eda-4cbb-a2d3-addbd4a22c08", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
58902025-04-30T06:43:54.430ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
58912025-04-30T06:43:54.448ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fe961d3b-0b8d-487d-8e41-2325ff9edaa3", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
58922025-04-30T06:43:54.449ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
58932025-04-30T06:43:54.452ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
58942025-04-30T06:43:54.458ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6390bd1e-2964-46da-8fcb-e36724ce5f23", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
58952025-04-30T06:43:54.459ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
58962025-04-30T06:43:54.460ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "930a3493-9145-4775-8041-97c76029c70f", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
58972025-04-30T06:43:54.466ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
58982025-04-30T06:43:54.470ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "16c94753-557c-41ff-b2ba-c0f8cecf22a5", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
58992025-04-30T06:43:54.470ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
59002025-04-30T06:43:54.501ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "bbb6f8fc-1c1a-4658-8f85-7358e6c5a39f", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
59012025-04-30T06:43:54.501ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
59022025-04-30T06:43:54.502ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6095a1f9-f6c1-4035-b816-b419a232dca8", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
59032025-04-30T06:43:54.502ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
59042025-04-30T06:43:54.502ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
59052025-04-30T06:43:54.502ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
59062025-04-30T06:43:54.502ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
59072025-04-30T06:43:54.504ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
59082025-04-30T06:43:54.504ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
59092025-04-30T06:43:54.504ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
59102025-04-30T06:43:54.504ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
59112025-04-30T06:43:54.504ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
59122025-04-30T06:43:54.504ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
59132025-04-30T06:43:54.508ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59142025-04-30T06:43:54.535ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
59152025-04-30T06:43:54.535ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
59162025-04-30T06:43:54.535ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
59172025-04-30T06:43:54.536ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
59182025-04-30T06:43:54.536ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
59192025-04-30T06:43:54.536ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
59202025-04-30T06:43:54.539ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59212025-04-30T06:43:54.544ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
59222025-04-30T06:43:54.544ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
59232025-04-30T06:43:54.544ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
59242025-04-30T06:43:54.545ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
59252025-04-30T06:43:54.546ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59262025-04-30T06:43:54.549ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
59272025-04-30T06:43:54.549ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
59282025-04-30T06:43:54.549ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
59292025-04-30T06:43:54.549ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
59302025-04-30T06:43:54.555ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
59312025-04-30T06:43:54.555ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
59322025-04-30T06:43:54.555ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
59332025-04-30T06:43:54.840ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
59342025-04-30T06:43:54.841ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory
59352025-04-30T06:43:54.843ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:65225 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
59362025-04-30T06:43:54.845ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:65225 req_id = b7d6d838-7251-4c4f-bc69-47f71a12a694 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
59372025-04-30T06:43:54.846ZINFOtest_unauthorized (dropshot): request completed latency_us = 1525 local_addr = [::1]:57786 method = GET remote_addr = [::1]:65225 req_id = b7d6d838-7251-4c4f-bc69-47f71a12a694 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
59382025-04-30T06:43:54.847ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = http://[::1]:57786 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:57786/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "b7d6d838-7251-4c4f-bc69-47f71a12a694", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
59392025-04-30T06:43:54.850ZDEBGtest_unauthorized (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
59402025-04-30T06:43:54.850ZDEBGtest_unauthorized (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
59412025-04-30T06:43:54.850ZDEBGtest_unauthorized (BackgroundTasks): finished collection background_task = inventory_collection
59422025-04-30T06:43:54.854ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
59432025-04-30T06:43:54.854ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
59442025-04-30T06:43:54.854ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
59452025-04-30T06:43:54.854ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
59462025-04-30T06:43:54.855ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
59472025-04-30T06:43:54.871ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59482025-04-30T06:43:54.876ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
59492025-04-30T06:43:54.876ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
59502025-04-30T06:43:54.876ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
59512025-04-30T06:43:54.877ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
59522025-04-30T06:43:54.888ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
59532025-04-30T06:43:54.957ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
59542025-04-30T06:43:54.957ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
59552025-04-30T06:43:54.957ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
59562025-04-30T06:43:54.957ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
59572025-04-30T06:43:54.959ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
59582025-04-30T06:43:54.961ZWARNtest_unauthorized (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
59592025-04-30T06:43:54.961ZWARNtest_unauthorized (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
59602025-04-30T06:43:54.962ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 903.638931ms iteration = 16 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
59612025-04-30T06:43:54.962ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 17 reason = Signaled
59622025-04-30T06:43:54.962ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
59632025-04-30T06:43:54.962ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
59642025-04-30T06:43:54.962ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
59652025-04-30T06:43:54.962ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
59662025-04-30T06:43:54.962ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
59672025-04-30T06:43:54.985ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59682025-04-30T06:43:54.993ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
59692025-04-30T06:43:54.993ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
59702025-04-30T06:43:54.993ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
59712025-04-30T06:43:54.993ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
59722025-04-30T06:43:54.996ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
59732025-04-30T06:43:54.996ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
59742025-04-30T06:43:54.997ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
59752025-04-30T06:43:54.997ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59762025-04-30T06:43:55.002ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
59772025-04-30T06:43:55.003ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
59782025-04-30T06:43:55.003ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
59792025-04-30T06:43:55.003ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
59802025-04-30T06:43:55.010ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1a8b2318-abf8-406d-a389-112ab91a0de8, 0)]
59812025-04-30T06:43:55.010ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
59822025-04-30T06:43:55.011ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: dc a8 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:50464 req_id = ca9817e5-8926-426b-8eb9-e16ca65fcff4
59832025-04-30T06:43:55.011ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 56488,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:50464 req_id = ca9817e5-8926-426b-8eb9-e16ca65fcff4
59842025-04-30T06:43:55.011ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
59852025-04-30T06:43:55.011ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
59862025-04-30T06:43:55.012ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:50464 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = ca9817e5-8926-426b-8eb9-e16ca65fcff4
59872025-04-30T06:43:55.012ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4299, tv_nsec: 811452215 } })
59882025-04-30T06:43:55.012ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
59892025-04-30T06:43:55.012ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 65 42 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e eB..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:49592 req_id = 3ff9c863-84ba-4f39-ad4f-8184997c2668
59902025-04-30T06:43:55.013ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 25922,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:49592 req_id = 3ff9c863-84ba-4f39-ad4f-8184997c2668
59912025-04-30T06:43:55.013ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
59922025-04-30T06:43:55.013ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:49592 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 3ff9c863-84ba-4f39-ad4f-8184997c2668
59932025-04-30T06:43:55.119ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
59942025-04-30T06:43:55.120ZINFOtest_unauthorized (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
59952025-04-30T06:43:55.121ZDEBGtest_unauthorized (oximeter): client request body = None method = GET uri = http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
59962025-04-30T06:43:55.122ZTRACtest_unauthorized (dropshot_internal): incoming request local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 630c1fd5-7d48-4542-a47f-e30c87849675 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
59972025-04-30T06:43:55.122ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 84 0c 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:65516 req_id = 8978c79c-ce4e-49f9-9cab-d5f239502c14
59982025-04-30T06:43:55.134ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 33804,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:65516 req_id = 8978c79c-ce4e-49f9-9cab-d5f239502c14
59992025-04-30T06:43:55.134ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
60002025-04-30T06:43:55.134ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 33804, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:65516 req_id = 8978c79c-ce4e-49f9-9cab-d5f239502c14
60012025-04-30T06:43:55.135ZTRACtest_unauthorized (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 630c1fd5-7d48-4542-a47f-e30c87849675 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
60022025-04-30T06:43:55.135ZDEBGtest_unauthorized (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 630c1fd5-7d48-4542-a47f-e30c87849675 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
60032025-04-30T06:43:55.137ZDEBGtest_unauthorized (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 630c1fd5-7d48-4542-a47f-e30c87849675 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
60042025-04-30T06:43:55.139ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
60052025-04-30T06:43:55.148ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
60062025-04-30T06:43:55.150ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
60072025-04-30T06:43:55.157ZINFOtest_unauthorized (dropshot_internal): request completed latency_us = 35140 local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 630c1fd5-7d48-4542-a47f-e30c87849675 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
60082025-04-30T06:43:55.158ZDEBGtest_unauthorized (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "630c1fd5-7d48-4542-a47f-e30c87849675", "content-length": "405", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
60092025-04-30T06:43:55.159ZDEBGtest_unauthorized (oximeter): client request body = None method = GET uri = http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
60102025-04-30T06:43:55.162ZTRACtest_unauthorized (dropshot_internal): incoming request local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = cf0ee0a3-7df7-4c1f-ac31-6b851a935f5c uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
60112025-04-30T06:43:55.162ZTRACtest_unauthorized (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = cf0ee0a3-7df7-4c1f-ac31-6b851a935f5c resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
60122025-04-30T06:43:55.162ZDEBGtest_unauthorized (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = cf0ee0a3-7df7-4c1f-ac31-6b851a935f5c roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
60132025-04-30T06:43:55.163ZDEBGtest_unauthorized (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = cf0ee0a3-7df7-4c1f-ac31-6b851a935f5c resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
60142025-04-30T06:43:55.165ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "bd7dcd10-4409-474c-ab64-e73413723dfe", "content-length": "835", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
60152025-04-30T06:43:55.165ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
60162025-04-30T06:43:55.167ZINFOtest_unauthorized (dropshot_internal): request completed latency_us = 5488 local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = cf0ee0a3-7df7-4c1f-ac31-6b851a935f5c response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
60172025-04-30T06:43:55.198ZDEBGtest_unauthorized (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "cf0ee0a3-7df7-4c1f-ac31-6b851a935f5c", "content-length": "29", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
60182025-04-30T06:43:55.204ZTRACtest_unauthorized (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 8d752adb-08b9-405f-a11b-66019bf447b5
60192025-04-30T06:43:55.204ZTRACtest_unauthorized (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
60202025-04-30T06:43:55.204ZINFOtest_unauthorized (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
60212025-04-30T06:43:55.205ZDEBGtest_unauthorized (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
60222025-04-30T06:43:55.205ZDEBGtest_unauthorized (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
60232025-04-30T06:43:55.205ZDEBGtest_unauthorized (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
60242025-04-30T06:43:55.207ZDEBGtest_unauthorized (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
60252025-04-30T06:43:55.208ZDEBGtest_unauthorized (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a
60262025-04-30T06:43:55.209ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
60272025-04-30T06:43:55.211ZTRACtest_unauthorized (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
60282025-04-30T06:43:55.212ZDEBGtest_unauthorized (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
60292025-04-30T06:43:55.212ZTRACtest_unauthorized (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
60302025-04-30T06:43:55.212ZTRACtest_unauthorized (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
60312025-04-30T06:43:55.213ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bd47072f-0338-4995-b442-c780174197b4", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
60322025-04-30T06:43:55.214ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
60332025-04-30T06:43:55.216ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 2 table_name = fields_string
60342025-04-30T06:43:55.216ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
60352025-04-30T06:43:55.220ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "47589900-bcff-4e09-836a-4a771ebcb1fc", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
60362025-04-30T06:43:55.220ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
60372025-04-30T06:43:55.224ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 1 table_name = measurements_i64
60382025-04-30T06:43:55.225ZTRACtest_unauthorized (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
60392025-04-30T06:43:55.226ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
60402025-04-30T06:43:55.229ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "12a12870-91dd-42a0-82d8-bfc13769e26f", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
60412025-04-30T06:43:55.229ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
60422025-04-30T06:43:55.231ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3bb25aeb-7412-4da7-803f-699809064212", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
60432025-04-30T06:43:55.231ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
60442025-04-30T06:43:55.233ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5cbf0cc2-a51f-4cb0-8615-a6300f0876f0", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
60452025-04-30T06:43:55.234ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
60462025-04-30T06:43:55.236ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e53bcd6e-166a-446e-9e6d-e03b87022939", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
60472025-04-30T06:43:55.236ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
60482025-04-30T06:43:55.238ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4a07a21e-d13e-4b04-9735-0d6eb1885693", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
60492025-04-30T06:43:55.238ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
60502025-04-30T06:43:55.239ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7f8fcf53-15be-4d90-882e-4326346fbd2e", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
60512025-04-30T06:43:55.239ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
60522025-04-30T06:43:55.240ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c852904d-6b86-406b-98a3-319a7d8f80d2", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
60532025-04-30T06:43:55.240ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
60542025-04-30T06:43:55.241ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "812c1a9b-a329-4d4a-9d23-c00919c7b588", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
60552025-04-30T06:43:55.241ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
60562025-04-30T06:43:55.242ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "24ffa23f-2a23-4cfc-9788-95e4cf4df7b3", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
60572025-04-30T06:43:55.242ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
60582025-04-30T06:43:55.245ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "332b79b1-d757-4a35-83f4-b5a6488f66af", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
60592025-04-30T06:43:55.246ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
60602025-04-30T06:43:55.251ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d538cd0b-c423-455e-b761-6670b7a90d6a", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
60612025-04-30T06:43:55.251ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
60622025-04-30T06:43:55.254ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8d50031a-b1c8-4c91-b6c1-2ae1e6d66114", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
60632025-04-30T06:43:55.254ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
60642025-04-30T06:43:55.256ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "45fed7cb-cee0-4f87-8151-f0b88a0ab5fe", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
60652025-04-30T06:43:55.256ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
60662025-04-30T06:43:55.258ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "468e7351-558f-4787-bb01-fa5c643c68b0", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
60672025-04-30T06:43:55.258ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
60682025-04-30T06:43:55.260ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f2cda7be-26eb-45dc-bf65-55b3ace22cb7", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
60692025-04-30T06:43:55.261ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
60702025-04-30T06:43:55.263ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d29f96fc-be0e-44a9-bafd-082378ac5b9c", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
60712025-04-30T06:43:55.263ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
60722025-04-30T06:43:55.266ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f602292f-39ab-43d3-bf67-4f5f4d6fc67b", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
60732025-04-30T06:43:55.267ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
60742025-04-30T06:43:55.268ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "83bbaf52-030e-4a35-83a1-9e5e61712aaf", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
60752025-04-30T06:43:55.268ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
60762025-04-30T06:43:55.269ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "860cc89e-de2b-491d-b62d-4334a907a8c4", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
60772025-04-30T06:43:55.269ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
60782025-04-30T06:43:55.270ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6c7d787d-7257-40b4-85f0-ed75ff0adfe7", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
60792025-04-30T06:43:55.270ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
60802025-04-30T06:43:55.271ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6e4470c2-7555-4560-a3c5-7db9e90a7c1c", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:54 GMT"} })
60812025-04-30T06:43:55.272ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
60822025-04-30T06:43:55.275ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "73a000f7-ed73-4207-8128-95bb7324d654", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
60832025-04-30T06:43:55.276ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
60842025-04-30T06:43:55.277ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
60852025-04-30T06:43:55.277ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
60862025-04-30T06:43:55.277ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
60872025-04-30T06:43:55.277ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
60882025-04-30T06:43:55.278ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
60892025-04-30T06:43:55.280ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60902025-04-30T06:43:55.285ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
60912025-04-30T06:43:55.286ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
60922025-04-30T06:43:55.286ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
60932025-04-30T06:43:55.286ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
60942025-04-30T06:43:55.287ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "07b04b37-ae59-4716-b823-756d61e83a77", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
60952025-04-30T06:43:55.287ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
60962025-04-30T06:43:55.289ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
60972025-04-30T06:43:55.289ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
60982025-04-30T06:43:55.290ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
60992025-04-30T06:43:55.291ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1df578d7-5725-4664-bdf5-6e5257cfc64c", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
61002025-04-30T06:43:55.291ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
61012025-04-30T06:43:55.293ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "56514658-ca4a-4e11-bbe4-b56bdebb2591", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
61022025-04-30T06:43:55.293ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
61032025-04-30T06:43:55.297ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b5661d95-0ec0-4383-b87e-b36b490d6d2c", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
61042025-04-30T06:43:55.297ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
61052025-04-30T06:43:55.303ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "02148919-30c4-402a-9c93-07346750a233", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
61062025-04-30T06:43:55.304ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
61072025-04-30T06:43:55.309ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9e9c1c3f-35d7-407d-b8c0-2caec932a9c2", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
61082025-04-30T06:43:55.309ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
61092025-04-30T06:43:55.310ZWARNtest_unauthorized (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
61102025-04-30T06:43:55.312ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0a0d4325-f620-4838-80cf-e42393f9bfde", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
61112025-04-30T06:43:55.312ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
61122025-04-30T06:43:55.314ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "64db9a36-d052-4dc6-885a-23b0aacc996e", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
61132025-04-30T06:43:55.315ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
61142025-04-30T06:43:55.316ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b7cd502b-b158-47b9-b4cd-80bdb9001914", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
61152025-04-30T06:43:55.316ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
61162025-04-30T06:43:55.318ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7e1a5039-9644-4e12-8e48-3918a02d80dd", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
61172025-04-30T06:43:55.318ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
61182025-04-30T06:43:55.320ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a834fd6a-fd1f-45bc-b5a0-36817019e194", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
61192025-04-30T06:43:55.320ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
61202025-04-30T06:43:55.322ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9b4b73e8-3938-485d-87cf-40e16aa2f693", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
61212025-04-30T06:43:55.322ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
61222025-04-30T06:43:55.327ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "40b42bd9-ce5e-41da-9cdd-eb36a83937f7", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
61232025-04-30T06:43:55.327ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
61242025-04-30T06:43:55.328ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
61252025-04-30T06:43:55.330ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "dc184c38-aa50-4eed-8955-8ae41b19918e", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
61262025-04-30T06:43:55.330ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
61272025-04-30T06:43:55.331ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a40e695-3d14-4069-968a-26fa24ffb5c6", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
61282025-04-30T06:43:55.332ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
61292025-04-30T06:43:55.333ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a966c4c-8655-4a0c-a258-cac9bc1fa67a", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
61302025-04-30T06:43:55.333ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
61312025-04-30T06:43:55.335ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8cb57c8f-d915-40ea-9857-a1e4ec3a34a0", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
61322025-04-30T06:43:55.335ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
61332025-04-30T06:43:55.336ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d6d3b545-35ee-4560-a90e-319aba1803c9", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
61342025-04-30T06:43:55.336ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
61352025-04-30T06:43:55.337ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2727a619-2976-4722-a88c-c434f8f1dbe1", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
61362025-04-30T06:43:55.337ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
61372025-04-30T06:43:55.338ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b6ce5f49-2d8b-403b-b0bb-2e8298de96f8", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
61382025-04-30T06:43:55.338ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
61392025-04-30T06:43:55.338ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
61402025-04-30T06:43:55.338ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
61412025-04-30T06:43:55.338ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
61422025-04-30T06:43:55.339ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
61432025-04-30T06:43:55.344ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61442025-04-30T06:43:55.347ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
61452025-04-30T06:43:55.347ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
61462025-04-30T06:43:55.347ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
61472025-04-30T06:43:55.347ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
61482025-04-30T06:43:55.348ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
61492025-04-30T06:43:55.348ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
61502025-04-30T06:43:55.351ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61512025-04-30T06:43:55.355ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
61522025-04-30T06:43:55.356ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
61532025-04-30T06:43:55.356ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
61542025-04-30T06:43:55.356ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
61552025-04-30T06:43:55.410ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
61562025-04-30T06:43:55.410ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory
61572025-04-30T06:43:55.411ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
61582025-04-30T06:43:55.411ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
61592025-04-30T06:43:55.411ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
61602025-04-30T06:43:55.411ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
61612025-04-30T06:43:55.413ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
61622025-04-30T06:43:55.413ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:46998 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
61632025-04-30T06:43:55.415ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:46998 req_id = 4c77769c-489e-475a-8bcd-dda4523c08d2 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
61642025-04-30T06:43:55.415ZINFOtest_unauthorized (dropshot): request completed latency_us = 448 local_addr = [::1]:57786 method = GET remote_addr = [::1]:46998 req_id = 4c77769c-489e-475a-8bcd-dda4523c08d2 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
61652025-04-30T06:43:55.416ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = http://[::1]:57786 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:57786/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "4c77769c-489e-475a-8bcd-dda4523c08d2", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
61662025-04-30T06:43:55.420ZDEBGtest_unauthorized (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
61672025-04-30T06:43:55.420ZDEBGtest_unauthorized (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
61682025-04-30T06:43:55.420ZDEBGtest_unauthorized (BackgroundTasks): finished collection background_task = inventory_collection
61692025-04-30T06:43:55.423ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
61702025-04-30T06:43:55.423ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
61712025-04-30T06:43:55.423ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
61722025-04-30T06:43:55.423ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
61732025-04-30T06:43:55.424ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
61742025-04-30T06:43:55.424ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61752025-04-30T06:43:55.431ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
61762025-04-30T06:43:55.438ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
61772025-04-30T06:43:55.438ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
61782025-04-30T06:43:55.438ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
61792025-04-30T06:43:55.441ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61802025-04-30T06:43:55.445ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
61812025-04-30T06:43:55.445ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
61822025-04-30T06:43:55.445ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
61832025-04-30T06:43:55.445ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
61842025-04-30T06:43:55.464ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
61852025-04-30T06:43:55.464ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
61862025-04-30T06:43:55.465ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
61872025-04-30T06:43:55.494ZWARNtest_unauthorized (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
61882025-04-30T06:43:55.495ZWARNtest_unauthorized (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
61892025-04-30T06:43:55.495ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 533.542409ms iteration = 17 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
61902025-04-30T06:43:55.495ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 18 reason = Signaled
61912025-04-30T06:43:55.495ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
61922025-04-30T06:43:55.495ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
61932025-04-30T06:43:55.495ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
61942025-04-30T06:43:55.495ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
61952025-04-30T06:43:55.496ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
61962025-04-30T06:43:55.501ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61972025-04-30T06:43:55.505ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
61982025-04-30T06:43:55.505ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
61992025-04-30T06:43:55.505ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
62002025-04-30T06:43:55.506ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
62012025-04-30T06:43:55.510ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
62022025-04-30T06:43:55.511ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1a8b2318-abf8-406d-a389-112ab91a0de8, 0)]
62032025-04-30T06:43:55.511ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
62042025-04-30T06:43:55.511ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 10 02 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:55743 req_id = fcc75a95-4253-4eb4-b1c8-3cacc7eecf1a
62052025-04-30T06:43:55.512ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 4098,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:55743 req_id = fcc75a95-4253-4eb4-b1c8-3cacc7eecf1a
62062025-04-30T06:43:55.512ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
62072025-04-30T06:43:55.512ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
62082025-04-30T06:43:55.512ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:55743 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = fcc75a95-4253-4eb4-b1c8-3cacc7eecf1a
62092025-04-30T06:43:55.514ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4300, tv_nsec: 313790802 } })
62102025-04-30T06:43:55.515ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
62112025-04-30T06:43:55.515ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 04 a5 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:60312 req_id = d6243f9d-c7d5-495d-abf7-0dfb86089e4c
62122025-04-30T06:43:55.515ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 1189,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60312 req_id = d6243f9d-c7d5-495d-abf7-0dfb86089e4c
62132025-04-30T06:43:55.516ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
62142025-04-30T06:43:55.516ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:60312 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = d6243f9d-c7d5-495d-abf7-0dfb86089e4c
62152025-04-30T06:43:55.584ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
62162025-04-30T06:43:55.585ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
62172025-04-30T06:43:55.585ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
62182025-04-30T06:43:55.586ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
62192025-04-30T06:43:55.586ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
62202025-04-30T06:43:55.587ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
62212025-04-30T06:43:55.600ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 38 a5 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c 8............_cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:61574 req_id = a3de6516-6eb2-45f8-9e4a-cbe17f6f7444
62222025-04-30T06:43:55.605ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 14501,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:61574 req_id = a3de6516-6eb2-45f8-9e4a-cbe17f6f7444
62232025-04-30T06:43:55.606ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
62242025-04-30T06:43:55.606ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 14501, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:61574 req_id = a3de6516-6eb2-45f8-9e4a-cbe17f6f7444
62252025-04-30T06:43:55.609ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
62262025-04-30T06:43:55.609ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
62272025-04-30T06:43:55.610ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
62282025-04-30T06:43:55.610ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62292025-04-30T06:43:55.625ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
62302025-04-30T06:43:55.625ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
62312025-04-30T06:43:55.625ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
62322025-04-30T06:43:55.626ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
62332025-04-30T06:43:55.642ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
62342025-04-30T06:43:55.642ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
62352025-04-30T06:43:55.643ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
62362025-04-30T06:43:55.647ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "f7c1829b-cfdc-4b33-a764-e0f368a39c10", "content-length": "835", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
62372025-04-30T06:43:55.647ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
62382025-04-30T06:43:55.673ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f9265982-371f-41dc-a18e-d2d3eee5f81b", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
62392025-04-30T06:43:55.674ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
62402025-04-30T06:43:55.693ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c8f0b971-e48a-4920-a234-8963b3250eba", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
62412025-04-30T06:43:55.693ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
62422025-04-30T06:43:55.699ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7ed5f67d-e168-463b-95e6-04b88a5f6e9c", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
62432025-04-30T06:43:55.699ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
62442025-04-30T06:43:55.702ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
62452025-04-30T06:43:55.704ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6cc201a3-21f8-4df1-aaba-2b03ba033c7f", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
62462025-04-30T06:43:55.704ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
62472025-04-30T06:43:55.712ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "475a28b5-01eb-4cfe-8a8c-aaffa100bf43", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
62482025-04-30T06:43:55.712ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
62492025-04-30T06:43:55.715ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "78eadd9d-806c-4683-9059-c00fbecabcb9", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
62502025-04-30T06:43:55.715ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
62512025-04-30T06:43:55.719ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2edcf35c-64c5-484a-a908-6e5694ebd709", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
62522025-04-30T06:43:55.720ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
62532025-04-30T06:43:55.721ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c18f86b0-f328-4529-a29e-e8f4fec05f1e", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
62542025-04-30T06:43:55.722ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
62552025-04-30T06:43:55.723ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3b603fd6-a706-4cc6-87e2-db591e506b7d", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
62562025-04-30T06:43:55.723ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
62572025-04-30T06:43:55.725ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d758c8d3-6621-442c-9c6f-5ae6894cee14", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
62582025-04-30T06:43:55.725ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
62592025-04-30T06:43:55.728ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3672c72d-78cb-4a9c-8b02-fee1474920f5", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
62602025-04-30T06:43:55.728ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
62612025-04-30T06:43:55.732ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "87f15a86-f0f4-4b1b-b807-a52584b01630", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
62622025-04-30T06:43:55.733ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
62632025-04-30T06:43:55.737ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d79a4940-78da-4597-9a49-abf3d042ec3c", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
62642025-04-30T06:43:55.738ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
62652025-04-30T06:43:55.740ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "efe26385-88b3-4456-aa33-e0eee76a9a5e", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
62662025-04-30T06:43:55.740ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
62672025-04-30T06:43:55.745ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "27cde5f0-8fa3-4440-b50d-8cffbb4308ab", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
62682025-04-30T06:43:55.745ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
62692025-04-30T06:43:55.749ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ae895bf2-c498-4d83-acb9-8cfe9daf2a60", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
62702025-04-30T06:43:55.749ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
62712025-04-30T06:43:55.754ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
62722025-04-30T06:43:55.754ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
62732025-04-30T06:43:55.754ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
62742025-04-30T06:43:55.756ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
62752025-04-30T06:43:55.757ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
62762025-04-30T06:43:55.757ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0b7da17a-7e4c-4a23-97f3-44d386d86636", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
62772025-04-30T06:43:55.758ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
62782025-04-30T06:43:55.784ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62792025-04-30T06:43:55.800ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
62802025-04-30T06:43:55.800ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
62812025-04-30T06:43:55.800ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
62822025-04-30T06:43:55.803ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
62832025-04-30T06:43:55.807ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
62842025-04-30T06:43:55.807ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
62852025-04-30T06:43:55.809ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
62862025-04-30T06:43:55.809ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "990d544e-4177-4468-8cbb-8d9d8f90a13d", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
62872025-04-30T06:43:55.813ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
62882025-04-30T06:43:55.817ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e27ed3e7-6182-47dc-941e-a060e6981fa8", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
62892025-04-30T06:43:55.817ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
62902025-04-30T06:43:55.823ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "80a7d206-2dde-4706-9203-2ac222a58338", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
62912025-04-30T06:43:55.823ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
62922025-04-30T06:43:55.824ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a1faf13-db89-4395-8366-fee33bcd4d12", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
62932025-04-30T06:43:55.825ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
62942025-04-30T06:43:55.841ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "07f5cb7a-1a51-49df-b58f-bbdafac31f03", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
62952025-04-30T06:43:55.847ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
62962025-04-30T06:43:55.852ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0494d811-4a29-45d7-959f-ae794141caa0", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
62972025-04-30T06:43:55.853ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
62982025-04-30T06:43:55.860ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "84664f7b-1495-44a5-b468-d50c46b65257", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
62992025-04-30T06:43:55.863ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
63002025-04-30T06:43:55.886ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9ca65756-1276-4473-9436-67960b758873", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
63012025-04-30T06:43:55.886ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
63022025-04-30T06:43:55.887ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
63032025-04-30T06:43:55.890ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d74e5ffc-e7fd-4f6d-af87-17f35e197261", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
63042025-04-30T06:43:55.890ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
63052025-04-30T06:43:55.892ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0c1d72e3-1043-435f-a92f-61b788d5390a", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
63062025-04-30T06:43:55.892ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
63072025-04-30T06:43:55.895ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2ad7453f-df11-4688-a773-0e7baad67704", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
63082025-04-30T06:43:55.895ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
63092025-04-30T06:43:55.898ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6713f043-b65d-4d04-b3d6-26654caababc", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
63102025-04-30T06:43:55.898ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
63112025-04-30T06:43:55.899ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb769c44-63a5-4759-b55c-353b5f0a4242", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
63122025-04-30T06:43:55.899ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
63132025-04-30T06:43:55.900ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1c8d4de4-7b8a-4e86-a791-433102d011f9", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
63142025-04-30T06:43:55.901ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
63152025-04-30T06:43:55.902ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1824d5da-5075-46a1-8f31-c91aa527ea64", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
63162025-04-30T06:43:55.902ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
63172025-04-30T06:43:55.904ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2eb42ffb-f7bc-427e-9595-271f780b6c88", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
63182025-04-30T06:43:55.908ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
63192025-04-30T06:43:55.916ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e396d5a6-d913-4765-90e6-e70c8ceed729", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
63202025-04-30T06:43:55.916ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
63212025-04-30T06:43:55.920ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6e42fe53-94e8-4507-9191-41b8affe6684", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
63222025-04-30T06:43:55.920ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
63232025-04-30T06:43:55.923ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ee76877e-4ec6-42a5-bf22-28b789bbebf1", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
63242025-04-30T06:43:55.923ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
63252025-04-30T06:43:55.925ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d1159a11-1100-46b2-8fd3-f30a88fcca59", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
63262025-04-30T06:43:55.926ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
63272025-04-30T06:43:55.928ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3e0637a6-3b65-4112-9228-8015e4bb71b4", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
63282025-04-30T06:43:55.928ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
63292025-04-30T06:43:55.930ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ac8e4317-6f4b-4406-b680-67c373a2a817", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
63302025-04-30T06:43:55.930ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
63312025-04-30T06:43:55.942ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
63322025-04-30T06:43:55.942ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
63332025-04-30T06:43:55.943ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
63342025-04-30T06:43:55.943ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
63352025-04-30T06:43:55.947ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
63362025-04-30T06:43:55.947ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "020cd07a-19fd-4ecc-bcf3-84eca19b234e", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
63372025-04-30T06:43:55.947ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
63382025-04-30T06:43:55.950ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2a72d8e8-03c8-4c69-a9d1-b91d234a0c05", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
63392025-04-30T06:43:55.952ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
63402025-04-30T06:43:55.952ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63412025-04-30T06:43:55.962ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
63422025-04-30T06:43:55.962ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
63432025-04-30T06:43:55.962ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
63442025-04-30T06:43:55.963ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
63452025-04-30T06:43:55.967ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
63462025-04-30T06:43:55.967ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
63472025-04-30T06:43:55.967ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
63482025-04-30T06:43:55.968ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "026e1d0f-8e7f-44c6-80bd-e185ece53c9b", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
63492025-04-30T06:43:55.970ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
63502025-04-30T06:43:55.974ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6bf20ad4-c6ab-4920-9ced-6822a9cf19fb", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
63512025-04-30T06:43:55.974ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
63522025-04-30T06:43:55.976ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c1196e34-62dd-4303-98e4-895c2e870429", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
63532025-04-30T06:43:55.976ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
63542025-04-30T06:43:55.977ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
63552025-04-30T06:43:55.977ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
63562025-04-30T06:43:55.977ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
63572025-04-30T06:43:55.978ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
63582025-04-30T06:43:55.982ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63592025-04-30T06:43:55.990ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
63602025-04-30T06:43:55.990ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
63612025-04-30T06:43:55.991ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
63622025-04-30T06:43:55.991ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
63632025-04-30T06:43:55.991ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
63642025-04-30T06:43:55.994ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
63652025-04-30T06:43:55.997ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63662025-04-30T06:43:56.004ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
63672025-04-30T06:43:56.008ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
63682025-04-30T06:43:56.008ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
63692025-04-30T06:43:56.009ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
63702025-04-30T06:43:56.114ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
63712025-04-30T06:43:56.114ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory
63722025-04-30T06:43:56.115ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:44433 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
63732025-04-30T06:43:56.117ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:44433 req_id = d45fc32d-18a2-45e2-9e39-2e00726e0f05 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
63742025-04-30T06:43:56.117ZINFOtest_unauthorized (dropshot): request completed latency_us = 497 local_addr = [::1]:57786 method = GET remote_addr = [::1]:44433 req_id = d45fc32d-18a2-45e2-9e39-2e00726e0f05 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
63752025-04-30T06:43:56.118ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = http://[::1]:57786 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:57786/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "d45fc32d-18a2-45e2-9e39-2e00726e0f05", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
63762025-04-30T06:43:56.127ZDEBGtest_unauthorized (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
63772025-04-30T06:43:56.127ZDEBGtest_unauthorized (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
63782025-04-30T06:43:56.127ZDEBGtest_unauthorized (BackgroundTasks): finished collection background_task = inventory_collection
63792025-04-30T06:43:56.129ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
63802025-04-30T06:43:56.129ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
63812025-04-30T06:43:56.129ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
63822025-04-30T06:43:56.129ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
63832025-04-30T06:43:56.129ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
63842025-04-30T06:43:56.134ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63852025-04-30T06:43:56.141ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
63862025-04-30T06:43:56.141ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
63872025-04-30T06:43:56.141ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
63882025-04-30T06:43:56.142ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
63892025-04-30T06:43:56.152ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
63902025-04-30T06:43:56.175ZDEBGtest_unauthorized (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
63912025-04-30T06:43:56.177ZDEBGtest_unauthorized (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
63922025-04-30T06:43:56.177ZDEBGtest_unauthorized (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a
63932025-04-30T06:43:56.178ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
63942025-04-30T06:43:56.191ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 2 table_name = fields_string
63952025-04-30T06:43:56.191ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
63962025-04-30T06:43:56.195ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 1 table_name = measurements_i64
63972025-04-30T06:43:56.195ZTRACtest_unauthorized (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
63982025-04-30T06:43:56.226ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
63992025-04-30T06:43:56.226ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
64002025-04-30T06:43:56.226ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
64012025-04-30T06:43:56.226ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
64022025-04-30T06:43:56.227ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
64032025-04-30T06:43:56.227ZDEBGtest_unauthorized (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
64042025-04-30T06:43:56.227ZDEBGtest_unauthorized (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
64052025-04-30T06:43:56.232ZTRACtest_unauthorized (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
64062025-04-30T06:43:56.233ZDEBGtest_unauthorized (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
64072025-04-30T06:43:56.233ZTRACtest_unauthorized (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
64082025-04-30T06:43:56.233ZTRACtest_unauthorized (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
64092025-04-30T06:43:56.234ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64102025-04-30T06:43:56.239ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
64112025-04-30T06:43:56.240ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
64122025-04-30T06:43:56.240ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
64132025-04-30T06:43:56.240ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
64142025-04-30T06:43:56.242ZWARNtest_unauthorized (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
64152025-04-30T06:43:56.243ZWARNtest_unauthorized (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
64162025-04-30T06:43:56.243ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 747.313412ms iteration = 18 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
64172025-04-30T06:43:56.243ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 19 reason = Signaled
64182025-04-30T06:43:56.243ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
64192025-04-30T06:43:56.243ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
64202025-04-30T06:43:56.243ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
64212025-04-30T06:43:56.243ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
64222025-04-30T06:43:56.245ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
64232025-04-30T06:43:56.245ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
64242025-04-30T06:43:56.245ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
64252025-04-30T06:43:56.247ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
64262025-04-30T06:43:56.249ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64272025-04-30T06:43:56.255ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
64282025-04-30T06:43:56.255ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
64292025-04-30T06:43:56.255ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
64302025-04-30T06:43:56.255ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
64312025-04-30T06:43:56.261ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1a8b2318-abf8-406d-a389-112ab91a0de8, 0)]
64322025-04-30T06:43:56.261ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
64332025-04-30T06:43:56.278ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 9b 4e 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .N..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:56355 req_id = 6e1f1b09-3527-4fb3-8f6b-967cb45028a8
64342025-04-30T06:43:56.279ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 39758,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:56355 req_id = 6e1f1b09-3527-4fb3-8f6b-967cb45028a8
64352025-04-30T06:43:56.279ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
64362025-04-30T06:43:56.279ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
64372025-04-30T06:43:56.279ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:56355 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 6e1f1b09-3527-4fb3-8f6b-967cb45028a8
64382025-04-30T06:43:56.280ZWARNtest_unauthorized (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
64392025-04-30T06:43:56.281ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4301, tv_nsec: 80589498 } })
64402025-04-30T06:43:56.281ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
64412025-04-30T06:43:56.282ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 2e dc 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:49949 req_id = dac28961-02dd-43f6-a6f1-1b0b5626af2b
64422025-04-30T06:43:56.283ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 11996,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:49949 req_id = dac28961-02dd-43f6-a6f1-1b0b5626af2b
64432025-04-30T06:43:56.283ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
64442025-04-30T06:43:56.283ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:49949 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = dac28961-02dd-43f6-a6f1-1b0b5626af2b
64452025-04-30T06:43:56.357ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
64462025-04-30T06:43:56.358ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 49 42 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c IB..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:50837 req_id = 82bf0234-a2fb-4ff5-a0b0-c08476014788
64472025-04-30T06:43:56.358ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 18754,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:50837 req_id = 82bf0234-a2fb-4ff5-a0b0-c08476014788
64482025-04-30T06:43:56.358ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
64492025-04-30T06:43:56.358ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 18754, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:50837 req_id = 82bf0234-a2fb-4ff5-a0b0-c08476014788
64502025-04-30T06:43:56.358ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
64512025-04-30T06:43:56.358ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
64522025-04-30T06:43:56.358ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
64532025-04-30T06:43:56.361ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "52d6f3e2-f35f-448b-aea6-632e9dc2deec", "content-length": "835", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
64542025-04-30T06:43:56.362ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
64552025-04-30T06:43:56.368ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "454e2495-cdf4-4545-ae39-5441a81e7c66", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
64562025-04-30T06:43:56.368ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
64572025-04-30T06:43:56.377ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7af68b62-789c-4ae4-8dfc-ad70cb7add9f", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
64582025-04-30T06:43:56.377ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
64592025-04-30T06:43:56.382ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "80837a02-7702-448d-ab5a-2ef329290df4", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
64602025-04-30T06:43:56.382ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
64612025-04-30T06:43:56.383ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
64622025-04-30T06:43:56.410ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f28a9185-d04b-4210-815d-e991be38718e", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
64632025-04-30T06:43:56.411ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
64642025-04-30T06:43:56.412ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2c99f3ba-2fb4-4d64-9e15-03d812442f73", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
64652025-04-30T06:43:56.412ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
64662025-04-30T06:43:56.415ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9f1e5dc8-d2bc-43ba-951b-52675695d9b7", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
64672025-04-30T06:43:56.416ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
64682025-04-30T06:43:56.418ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2c82e7b7-54ba-41af-a573-52e951d68b25", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
64692025-04-30T06:43:56.418ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
64702025-04-30T06:43:56.419ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b8f56ecb-14a5-46f1-8b75-d46578e09fdb", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
64712025-04-30T06:43:56.419ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
64722025-04-30T06:43:56.421ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a6240a0b-9d11-4906-8b9d-e64b6d9ddaeb", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
64732025-04-30T06:43:56.421ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
64742025-04-30T06:43:56.423ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ae869677-32ca-4776-bfa4-08301ed30bcd", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
64752025-04-30T06:43:56.423ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
64762025-04-30T06:43:56.424ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9bfcad5e-9c68-4eaf-852c-d375ad351d17", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
64772025-04-30T06:43:56.424ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
64782025-04-30T06:43:56.425ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a1602c6-8655-4958-8187-f7c20b73c793", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
64792025-04-30T06:43:56.425ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
64802025-04-30T06:43:56.427ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e09ec77f-1333-45b1-b202-2297e17843dd", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
64812025-04-30T06:43:56.428ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
64822025-04-30T06:43:56.430ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d096d5fc-f21a-4d04-ae40-23a7b2fce339", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
64832025-04-30T06:43:56.430ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
64842025-04-30T06:43:56.434ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "62cb5c0f-02dc-447d-af91-3b741ba036d9", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
64852025-04-30T06:43:56.434ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
64862025-04-30T06:43:56.437ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ba460e3b-cff2-4fc4-8e95-3c7c811ffcfa", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
64872025-04-30T06:43:56.437ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
64882025-04-30T06:43:56.439ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1bec358e-d075-4bd2-9e06-7ec53bb11f32", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
64892025-04-30T06:43:56.439ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
64902025-04-30T06:43:56.447ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
64912025-04-30T06:43:56.447ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
64922025-04-30T06:43:56.447ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
64932025-04-30T06:43:56.447ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
64942025-04-30T06:43:56.448ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
64952025-04-30T06:43:56.453ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8f65be83-e343-4f77-907e-927fba94e567", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
64962025-04-30T06:43:56.454ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
64972025-04-30T06:43:56.454ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64982025-04-30T06:43:56.460ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
64992025-04-30T06:43:56.461ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
65002025-04-30T06:43:56.461ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
65012025-04-30T06:43:56.461ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
65022025-04-30T06:43:56.465ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3a58a4b5-0478-43f0-9d01-3860fb618bb6", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
65032025-04-30T06:43:56.465ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
65042025-04-30T06:43:56.465ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
65052025-04-30T06:43:56.465ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
65062025-04-30T06:43:56.465ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
65072025-04-30T06:43:56.478ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6c687670-f878-49f3-8db3-36c764cdf045", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
65082025-04-30T06:43:56.478ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
65092025-04-30T06:43:56.480ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "26a37b05-c27d-4118-8e0c-4668bb23e42a", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
65102025-04-30T06:43:56.482ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
65112025-04-30T06:43:56.486ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4ffdcca1-4e88-45b1-9509-1dcd2fbf0216", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
65122025-04-30T06:43:56.486ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
65132025-04-30T06:43:56.504ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1d56744f-d629-49cf-8c17-14db9e16afab", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
65142025-04-30T06:43:56.504ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
65152025-04-30T06:43:56.523ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fd685229-d966-4128-b308-35ae1dbe0e50", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
65162025-04-30T06:43:56.523ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
65172025-04-30T06:43:56.525ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "29cff8c4-c4c3-4574-a306-8a1c56d38e33", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
65182025-04-30T06:43:56.525ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
65192025-04-30T06:43:56.530ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ee8efb54-3650-4256-9d6b-bf24f22854df", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
65202025-04-30T06:43:56.539ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
65212025-04-30T06:43:56.548ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
65222025-04-30T06:43:56.600ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
65232025-04-30T06:43:56.600ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
65242025-04-30T06:43:56.600ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
65252025-04-30T06:43:56.600ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
65262025-04-30T06:43:56.601ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
65272025-04-30T06:43:56.607ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "98e0fe04-c99d-4886-b3de-83cd46901220", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
65282025-04-30T06:43:56.607ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
65292025-04-30T06:43:56.607ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65302025-04-30T06:43:56.612ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
65312025-04-30T06:43:56.612ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
65322025-04-30T06:43:56.612ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
65332025-04-30T06:43:56.612ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
65342025-04-30T06:43:56.653ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
65352025-04-30T06:43:56.653ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
65362025-04-30T06:43:56.664ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
65372025-04-30T06:43:56.695ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "25682150-acc8-4d87-9fca-3533ed5ca18b", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
65382025-04-30T06:43:56.695ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
65392025-04-30T06:43:56.725ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4edcf726-c6f7-4576-9bb8-ce27de07aa8c", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
65402025-04-30T06:43:56.725ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
65412025-04-30T06:43:56.727ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "054653c4-8da8-43a0-9973-29d5a196f9cc", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
65422025-04-30T06:43:56.727ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
65432025-04-30T06:43:56.730ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "89017621-5896-47a3-b965-b62db51442e0", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
65442025-04-30T06:43:56.731ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
65452025-04-30T06:43:56.734ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f0d2fc28-6436-45b9-a952-6208a5f4b655", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
65462025-04-30T06:43:56.734ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
65472025-04-30T06:43:56.739ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c36089d6-3078-4113-bec3-9b9c08be8a0c", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
65482025-04-30T06:43:56.741ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
65492025-04-30T06:43:56.769ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
65502025-04-30T06:43:56.772ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "68c5d2e8-1964-4310-8575-6b66dc8f2932", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
65512025-04-30T06:43:56.772ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
65522025-04-30T06:43:56.777ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "451715e6-a977-4d71-8953-0a6b19f0a17a", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
65532025-04-30T06:43:56.777ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
65542025-04-30T06:43:56.789ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0b6eb69a-6e37-47cb-96b2-ddb42b69ce50", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
65552025-04-30T06:43:56.791ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
65562025-04-30T06:43:56.797ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f9f4bb58-bae5-4333-b02e-4c653cb3489f", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
65572025-04-30T06:43:56.798ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
65582025-04-30T06:43:56.803ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "39dc0497-ff37-43c5-a109-ae1732c39240", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
65592025-04-30T06:43:56.803ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
65602025-04-30T06:43:56.815ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "26993ab2-859d-4cba-9ab9-21255577eac8", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
65612025-04-30T06:43:56.815ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
65622025-04-30T06:43:56.820ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
65632025-04-30T06:43:56.820ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
65642025-04-30T06:43:56.820ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
65652025-04-30T06:43:56.820ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
65662025-04-30T06:43:56.821ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
65672025-04-30T06:43:56.824ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65682025-04-30T06:43:56.836ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
65692025-04-30T06:43:56.870ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
65702025-04-30T06:43:56.870ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
65712025-04-30T06:43:56.871ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
65722025-04-30T06:43:56.872ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "14da348f-7ada-4938-b948-32572f412c93", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
65732025-04-30T06:43:56.872ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
65742025-04-30T06:43:56.875ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c30c2651-e3d0-4387-aec4-f7896d15f219", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
65752025-04-30T06:43:56.876ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
65762025-04-30T06:43:56.876ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
65772025-04-30T06:43:56.877ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
65782025-04-30T06:43:56.877ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
65792025-04-30T06:43:56.879ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ce6f1fa5-304f-49f4-b471-18e63f50beac", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
65802025-04-30T06:43:56.879ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
65812025-04-30T06:43:56.883ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "90730167-6523-4d12-b389-3548932fa8a2", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
65822025-04-30T06:43:56.883ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
65832025-04-30T06:43:56.927ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1aa050b9-6d47-4edb-9b88-ea51a1ef2bbf", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
65842025-04-30T06:43:56.928ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
65852025-04-30T06:43:56.928ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
65862025-04-30T06:43:56.928ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
65872025-04-30T06:43:56.928ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
65882025-04-30T06:43:56.929ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
65892025-04-30T06:43:56.933ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65902025-04-30T06:43:56.940ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
65912025-04-30T06:43:56.940ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
65922025-04-30T06:43:56.940ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
65932025-04-30T06:43:56.940ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
65942025-04-30T06:43:56.940ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
65952025-04-30T06:43:56.941ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
65962025-04-30T06:43:56.945ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65972025-04-30T06:43:56.948ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
65982025-04-30T06:43:56.948ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
65992025-04-30T06:43:56.949ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
66002025-04-30T06:43:56.951ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
66012025-04-30T06:43:57.055ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
66022025-04-30T06:43:57.055ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory
66032025-04-30T06:43:57.055ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:61336 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
66042025-04-30T06:43:57.056ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:61336 req_id = c3c49555-8b6b-45ff-8814-b02dc84e7d0c sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
66052025-04-30T06:43:57.056ZINFOtest_unauthorized (dropshot): request completed latency_us = 538 local_addr = [::1]:57786 method = GET remote_addr = [::1]:61336 req_id = c3c49555-8b6b-45ff-8814-b02dc84e7d0c response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
66062025-04-30T06:43:57.057ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = http://[::1]:57786 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:57786/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "c3c49555-8b6b-45ff-8814-b02dc84e7d0c", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
66072025-04-30T06:43:57.059ZDEBGtest_unauthorized (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
66082025-04-30T06:43:57.059ZDEBGtest_unauthorized (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
66092025-04-30T06:43:57.059ZDEBGtest_unauthorized (BackgroundTasks): finished collection background_task = inventory_collection
66102025-04-30T06:43:57.060ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
66112025-04-30T06:43:57.060ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
66122025-04-30T06:43:57.060ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
66132025-04-30T06:43:57.060ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
66142025-04-30T06:43:57.069ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
66152025-04-30T06:43:57.076ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66162025-04-30T06:43:57.080ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
66172025-04-30T06:43:57.080ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
66182025-04-30T06:43:57.080ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
66192025-04-30T06:43:57.081ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
66202025-04-30T06:43:57.090ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
66212025-04-30T06:43:57.099ZINFOtest_unauthorized (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
66222025-04-30T06:43:57.125ZDEBGtest_unauthorized (oximeter): client request body = None method = GET uri = http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
66232025-04-30T06:43:57.126ZTRACtest_unauthorized (dropshot_internal): incoming request local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 08c91d53-a910-4a0f-b306-7dae213f0384 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
66242025-04-30T06:43:57.126ZTRACtest_unauthorized (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 08c91d53-a910-4a0f-b306-7dae213f0384 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
66252025-04-30T06:43:57.126ZDEBGtest_unauthorized (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 08c91d53-a910-4a0f-b306-7dae213f0384 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
66262025-04-30T06:43:57.127ZDEBGtest_unauthorized (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 08c91d53-a910-4a0f-b306-7dae213f0384 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
66272025-04-30T06:43:57.131ZINFOtest_unauthorized (dropshot_internal): request completed latency_us = 5509 local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 08c91d53-a910-4a0f-b306-7dae213f0384 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
66282025-04-30T06:43:57.133ZDEBGtest_unauthorized (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "08c91d53-a910-4a0f-b306-7dae213f0384", "content-length": "405", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
66292025-04-30T06:43:57.133ZDEBGtest_unauthorized (oximeter): client request body = None method = GET uri = http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
66302025-04-30T06:43:57.133ZTRACtest_unauthorized (dropshot_internal): incoming request local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 5eccebad-566c-4696-90ce-1462a172114d uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
66312025-04-30T06:43:57.134ZTRACtest_unauthorized (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 5eccebad-566c-4696-90ce-1462a172114d resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
66322025-04-30T06:43:57.134ZDEBGtest_unauthorized (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 5eccebad-566c-4696-90ce-1462a172114d roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
66332025-04-30T06:43:57.135ZDEBGtest_unauthorized (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 5eccebad-566c-4696-90ce-1462a172114d resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
66342025-04-30T06:43:57.152ZINFOtest_unauthorized (dropshot_internal): request completed latency_us = 18575 local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 5eccebad-566c-4696-90ce-1462a172114d response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
66352025-04-30T06:43:57.159ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
66362025-04-30T06:43:57.160ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
66372025-04-30T06:43:57.160ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
66382025-04-30T06:43:57.160ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
66392025-04-30T06:43:57.162ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
66402025-04-30T06:43:57.162ZDEBGtest_unauthorized (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "5eccebad-566c-4696-90ce-1462a172114d", "content-length": "29", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
66412025-04-30T06:43:57.162ZTRACtest_unauthorized (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 8d752adb-08b9-405f-a11b-66019bf447b5
66422025-04-30T06:43:57.162ZTRACtest_unauthorized (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
66432025-04-30T06:43:57.162ZINFOtest_unauthorized (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
66442025-04-30T06:43:57.165ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66452025-04-30T06:43:57.170ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
66462025-04-30T06:43:57.170ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
66472025-04-30T06:43:57.170ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
66482025-04-30T06:43:57.171ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
66492025-04-30T06:43:57.184ZDEBGtest_unauthorized (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
66502025-04-30T06:43:57.184ZDEBGtest_unauthorized (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
66512025-04-30T06:43:57.185ZDEBGtest_unauthorized (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a
66522025-04-30T06:43:57.186ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
66532025-04-30T06:43:57.188ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
66542025-04-30T06:43:57.188ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
66552025-04-30T06:43:57.189ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
66562025-04-30T06:43:57.190ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 2 table_name = fields_string
66572025-04-30T06:43:57.190ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
66582025-04-30T06:43:57.190ZWARNtest_unauthorized (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
66592025-04-30T06:43:57.191ZWARNtest_unauthorized (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
66602025-04-30T06:43:57.191ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 947.976861ms iteration = 19 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
66612025-04-30T06:43:57.191ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 20 reason = Signaled
66622025-04-30T06:43:57.191ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
66632025-04-30T06:43:57.191ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
66642025-04-30T06:43:57.191ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
66652025-04-30T06:43:57.191ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
66662025-04-30T06:43:57.191ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
66672025-04-30T06:43:57.195ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 1 table_name = measurements_i64
66682025-04-30T06:43:57.195ZTRACtest_unauthorized (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
66692025-04-30T06:43:57.196ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66702025-04-30T06:43:57.200ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
66712025-04-30T06:43:57.200ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
66722025-04-30T06:43:57.200ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
66732025-04-30T06:43:57.201ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
66742025-04-30T06:43:57.202ZDEBGtest_unauthorized (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
66752025-04-30T06:43:57.202ZDEBGtest_unauthorized (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
66762025-04-30T06:43:57.212ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1a8b2318-abf8-406d-a389-112ab91a0de8, 0)]
66772025-04-30T06:43:57.212ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
66782025-04-30T06:43:57.214ZTRACtest_unauthorized (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
66792025-04-30T06:43:57.214ZDEBGtest_unauthorized (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
66802025-04-30T06:43:57.214ZTRACtest_unauthorized (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
66812025-04-30T06:43:57.214ZTRACtest_unauthorized (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
66822025-04-30T06:43:57.216ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: cd 24 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .$..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:57044 req_id = 04400932-1a28-4c5d-ba0c-5ad3c0568bcf
66832025-04-30T06:43:57.216ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 52516,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:57044 req_id = 04400932-1a28-4c5d-ba0c-5ad3c0568bcf
66842025-04-30T06:43:57.216ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
66852025-04-30T06:43:57.216ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
66862025-04-30T06:43:57.216ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:57044 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 04400932-1a28-4c5d-ba0c-5ad3c0568bcf
66872025-04-30T06:43:57.217ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4302, tv_nsec: 16631340 } })
66882025-04-30T06:43:57.217ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
66892025-04-30T06:43:57.218ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 9b 97 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:59909 req_id = 0213a3fd-09bd-4f8c-b7f0-548fe2da0b49
66902025-04-30T06:43:57.218ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 39831,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:59909 req_id = 0213a3fd-09bd-4f8c-b7f0-548fe2da0b49
66912025-04-30T06:43:57.218ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
66922025-04-30T06:43:57.218ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:59909 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 0213a3fd-09bd-4f8c-b7f0-548fe2da0b49
66932025-04-30T06:43:57.268ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
66942025-04-30T06:43:57.270ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: ad dd 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:64803 req_id = e29b3464-f97e-446b-b4f2-fa13e80bf06a
66952025-04-30T06:43:57.271ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 44509,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64803 req_id = e29b3464-f97e-446b-b4f2-fa13e80bf06a
66962025-04-30T06:43:57.271ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
66972025-04-30T06:43:57.271ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 44509, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:64803 req_id = e29b3464-f97e-446b-b4f2-fa13e80bf06a
66982025-04-30T06:43:57.271ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
66992025-04-30T06:43:57.272ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
67002025-04-30T06:43:57.272ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
67012025-04-30T06:43:57.273ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "b45e8554-34af-4d81-be74-696f091296a7", "content-length": "835", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
67022025-04-30T06:43:57.274ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
67032025-04-30T06:43:57.276ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "51da297a-e412-4969-8ebb-ee4e440ba5ec", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
67042025-04-30T06:43:57.276ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
67052025-04-30T06:43:57.279ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8d2fa66b-3ad3-47d7-a3bd-223e8f6eca0c", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
67062025-04-30T06:43:57.279ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
67072025-04-30T06:43:57.285ZWARNtest_unauthorized (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
67082025-04-30T06:43:57.286ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
67092025-04-30T06:43:57.286ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "30ca90fa-4b0a-4ed1-a580-a47ba4a28cf3", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
67102025-04-30T06:43:57.286ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
67112025-04-30T06:43:57.304ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f72a8e98-d656-4f96-ae27-2c9f265325dd", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
67122025-04-30T06:43:57.304ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
67132025-04-30T06:43:57.307ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "48c22fb1-ef78-411c-b8b3-5b8892cb7d9b", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
67142025-04-30T06:43:57.307ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
67152025-04-30T06:43:57.310ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "582b73be-b52e-4e55-bc7d-eed5e8a9edf6", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
67162025-04-30T06:43:57.310ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
67172025-04-30T06:43:57.312ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "18c8b509-d2d9-4505-a29d-1b643e0e9493", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
67182025-04-30T06:43:57.312ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
67192025-04-30T06:43:57.313ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c5164138-49de-4ecc-9ff4-068ab2b2e075", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
67202025-04-30T06:43:57.314ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
67212025-04-30T06:43:57.315ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4386bb02-833c-4b49-a703-a9fb7ca5096f", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
67222025-04-30T06:43:57.315ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
67232025-04-30T06:43:57.316ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "065d217a-aa54-4745-a00c-8ad0a0ba1a2b", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
67242025-04-30T06:43:57.316ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
67252025-04-30T06:43:57.317ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6cc32974-b9a0-4ccb-ab0e-a0c288013f0f", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
67262025-04-30T06:43:57.320ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
67272025-04-30T06:43:57.321ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "004ee909-434d-4254-994d-a102ca66f08e", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
67282025-04-30T06:43:57.322ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
67292025-04-30T06:43:57.325ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9d7973d6-a11f-4104-bbb1-1e2d27a5322e", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
67302025-04-30T06:43:57.326ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
67312025-04-30T06:43:57.328ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9472be1e-a924-4ab7-89cb-09ad1c585e6e", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
67322025-04-30T06:43:57.328ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
67332025-04-30T06:43:57.330ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "69bef004-0632-471b-a9d5-8f781f0bf2fd", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
67342025-04-30T06:43:57.330ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
67352025-04-30T06:43:57.332ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9207a8a5-4aaf-4136-9601-48dda189c962", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
67362025-04-30T06:43:57.332ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
67372025-04-30T06:43:57.334ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "39f600ce-3fa6-4a14-9a49-0b2b4a612523", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
67382025-04-30T06:43:57.334ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
67392025-04-30T06:43:57.337ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "420b4b30-f60e-47d5-8ef9-cdab3eadc073", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
67402025-04-30T06:43:57.337ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
67412025-04-30T06:43:57.337ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
67422025-04-30T06:43:57.337ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
67432025-04-30T06:43:57.337ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
67442025-04-30T06:43:57.337ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
67452025-04-30T06:43:57.337ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
67462025-04-30T06:43:57.359ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c53bb077-ee6b-47b6-96ba-1213a59d0d74", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:56 GMT"} })
67472025-04-30T06:43:57.360ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
67482025-04-30T06:43:57.361ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "665919c4-906a-423b-852e-c082218ee6a2", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
67492025-04-30T06:43:57.361ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
67502025-04-30T06:43:57.364ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67512025-04-30T06:43:57.368ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
67522025-04-30T06:43:57.368ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
67532025-04-30T06:43:57.369ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
67542025-04-30T06:43:57.369ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
67552025-04-30T06:43:57.373ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ea5a0934-d471-4f39-a094-456992ac5ce4", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
67562025-04-30T06:43:57.373ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
67572025-04-30T06:43:57.374ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
67582025-04-30T06:43:57.374ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
67592025-04-30T06:43:57.375ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
67602025-04-30T06:43:57.376ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "55e093dc-bbfd-41c1-ba57-de0fa31e75f5", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
67612025-04-30T06:43:57.376ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
67622025-04-30T06:43:57.378ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "43bcda9b-b74b-4045-93d0-2ac726feeb69", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
67632025-04-30T06:43:57.378ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
67642025-04-30T06:43:57.380ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3c952068-08b0-47f5-a5ac-9dfe336ca890", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
67652025-04-30T06:43:57.380ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
67662025-04-30T06:43:57.384ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "21b5c915-2763-43b7-8129-97fd826d74a0", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
67672025-04-30T06:43:57.384ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
67682025-04-30T06:43:57.386ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "959af311-d996-458e-80ea-31acca253f2c", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
67692025-04-30T06:43:57.387ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
67702025-04-30T06:43:57.388ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9708afcb-ed6c-45f6-8ec5-7168fadcfa6a", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
67712025-04-30T06:43:57.388ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
67722025-04-30T06:43:57.391ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6d3894a6-775a-4851-8e42-1c61c20ca281", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
67732025-04-30T06:43:57.391ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
67742025-04-30T06:43:57.393ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6363434f-f112-4784-95f0-ae08a2af7073", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
67752025-04-30T06:43:57.393ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
67762025-04-30T06:43:57.394ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4b26c373-e630-47a6-bd07-5e4e1520f04d", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
67772025-04-30T06:43:57.394ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
67782025-04-30T06:43:57.396ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f6192af0-9d4a-4ea2-bbb8-1eada828f052", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
67792025-04-30T06:43:57.396ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
67802025-04-30T06:43:57.404ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f92c1124-9b40-4a52-b04b-1d778bb1239b", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
67812025-04-30T06:43:57.404ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
67822025-04-30T06:43:57.410ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "58039474-a90c-4784-9322-77f455aeee2c", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
67832025-04-30T06:43:57.410ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
67842025-04-30T06:43:57.413ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "eecf5767-b215-458a-8c07-4dbb60479892", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
67852025-04-30T06:43:57.413ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
67862025-04-30T06:43:57.414ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
67872025-04-30T06:43:57.416ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3e2c57b1-97a7-40a1-b8ce-48b200fbd21a", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
67882025-04-30T06:43:57.416ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
67892025-04-30T06:43:57.419ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "19876409-41dc-46b0-9293-d3d4afff4d18", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
67902025-04-30T06:43:57.419ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
67912025-04-30T06:43:57.421ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "188791ff-8d94-40f3-816b-c5ac70510acf", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
67922025-04-30T06:43:57.421ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
67932025-04-30T06:43:57.423ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ea2016bc-c69c-44bd-bc79-088a7de1f94c", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
67942025-04-30T06:43:57.423ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
67952025-04-30T06:43:57.425ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cf5581ec-8e2a-4a32-bc61-42ef7a44ca87", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
67962025-04-30T06:43:57.425ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
67972025-04-30T06:43:57.428ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7cfe744a-86fa-47a7-bbf6-ba4a0a51a007", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
67982025-04-30T06:43:57.428ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
67992025-04-30T06:43:57.429ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1d11c02e-d260-4542-b91a-c5b52bf6d6ec", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
68002025-04-30T06:43:57.429ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
68012025-04-30T06:43:57.430ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9724a5a7-e086-44a6-9d2a-aa3b4a6e5194", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
68022025-04-30T06:43:57.430ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
68032025-04-30T06:43:57.431ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "de0c2c17-e9c0-446c-a031-9475e729ab9a", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
68042025-04-30T06:43:57.431ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
68052025-04-30T06:43:57.432ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "69c7b225-ecdf-42b2-8fd9-c5e94aa3345f", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
68062025-04-30T06:43:57.432ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
68072025-04-30T06:43:57.433ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
68082025-04-30T06:43:57.433ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
68092025-04-30T06:43:57.433ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
68102025-04-30T06:43:57.434ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
68112025-04-30T06:43:57.442ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68122025-04-30T06:43:57.446ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
68132025-04-30T06:43:57.446ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
68142025-04-30T06:43:57.446ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
68152025-04-30T06:43:57.446ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
68162025-04-30T06:43:57.446ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
68172025-04-30T06:43:57.446ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
68182025-04-30T06:43:57.450ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68192025-04-30T06:43:57.455ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
68202025-04-30T06:43:57.455ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
68212025-04-30T06:43:57.456ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
68222025-04-30T06:43:57.456ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
68232025-04-30T06:43:57.526ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
68242025-04-30T06:43:57.526ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory
68252025-04-30T06:43:57.526ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
68262025-04-30T06:43:57.526ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
68272025-04-30T06:43:57.526ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
68282025-04-30T06:43:57.526ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
68292025-04-30T06:43:57.527ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
68302025-04-30T06:43:57.527ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:50741 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
68312025-04-30T06:43:57.529ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:50741 req_id = a620a237-17e0-447a-85ff-505d60c3a0f7 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
68322025-04-30T06:43:57.530ZINFOtest_unauthorized (dropshot): request completed latency_us = 903 local_addr = [::1]:57786 method = GET remote_addr = [::1]:50741 req_id = a620a237-17e0-447a-85ff-505d60c3a0f7 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
68332025-04-30T06:43:57.532ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = http://[::1]:57786 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:57786/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "a620a237-17e0-447a-85ff-505d60c3a0f7", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
68342025-04-30T06:43:57.535ZDEBGtest_unauthorized (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
68352025-04-30T06:43:57.535ZDEBGtest_unauthorized (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
68362025-04-30T06:43:57.535ZDEBGtest_unauthorized (BackgroundTasks): finished collection background_task = inventory_collection
68372025-04-30T06:43:57.537ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
68382025-04-30T06:43:57.537ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
68392025-04-30T06:43:57.538ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
68402025-04-30T06:43:57.538ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
68412025-04-30T06:43:57.538ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
68422025-04-30T06:43:57.539ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68432025-04-30T06:43:57.550ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
68442025-04-30T06:43:57.550ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
68452025-04-30T06:43:57.550ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
68462025-04-30T06:43:57.552ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
68472025-04-30T06:43:57.557ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68482025-04-30T06:43:57.561ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
68492025-04-30T06:43:57.561ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
68502025-04-30T06:43:57.561ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
68512025-04-30T06:43:57.562ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
68522025-04-30T06:43:57.562ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
68532025-04-30T06:43:57.562ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
68542025-04-30T06:43:57.564ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
68552025-04-30T06:43:57.622ZWARNtest_unauthorized (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
68562025-04-30T06:43:57.623ZWARNtest_unauthorized (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
68572025-04-30T06:43:57.623ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 432.579617ms iteration = 20 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
68582025-04-30T06:43:57.623ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 21 reason = Signaled
68592025-04-30T06:43:57.623ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
68602025-04-30T06:43:57.623ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
68612025-04-30T06:43:57.623ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
68622025-04-30T06:43:57.624ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
68632025-04-30T06:43:57.625ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
68642025-04-30T06:43:57.633ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68652025-04-30T06:43:57.638ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
68662025-04-30T06:43:57.638ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
68672025-04-30T06:43:57.638ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
68682025-04-30T06:43:57.639ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
68692025-04-30T06:43:57.643ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1a8b2318-abf8-406d-a389-112ab91a0de8, 0)]
68702025-04-30T06:43:57.643ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
68712025-04-30T06:43:57.644ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: a2 86 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:64365 req_id = efa0ea4b-8869-4857-9151-8e923a8199b4
68722025-04-30T06:43:57.644ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 41606,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64365 req_id = efa0ea4b-8869-4857-9151-8e923a8199b4
68732025-04-30T06:43:57.644ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
68742025-04-30T06:43:57.645ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
68752025-04-30T06:43:57.645ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:64365 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = efa0ea4b-8869-4857-9151-8e923a8199b4
68762025-04-30T06:43:57.645ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4302, tv_nsec: 444691402 } })
68772025-04-30T06:43:57.646ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
68782025-04-30T06:43:57.646ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: ea 10 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:62460 req_id = 600cb4b6-78d3-4ab1-9f5e-c94678fb8163
68792025-04-30T06:43:57.646ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 59920,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62460 req_id = 600cb4b6-78d3-4ab1-9f5e-c94678fb8163
68802025-04-30T06:43:57.646ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
68812025-04-30T06:43:57.647ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:62460 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 600cb4b6-78d3-4ab1-9f5e-c94678fb8163
68822025-04-30T06:43:57.705ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
68832025-04-30T06:43:57.707ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 8a f1 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:62166 req_id = 5d98af44-346b-4ac2-bd6e-a944b5451f76
68842025-04-30T06:43:57.707ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 35569,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62166 req_id = 5d98af44-346b-4ac2-bd6e-a944b5451f76
68852025-04-30T06:43:57.708ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
68862025-04-30T06:43:57.708ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 35569, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:62166 req_id = 5d98af44-346b-4ac2-bd6e-a944b5451f76
68872025-04-30T06:43:57.708ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
68882025-04-30T06:43:57.708ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
68892025-04-30T06:43:57.708ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
68902025-04-30T06:43:57.713ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
68912025-04-30T06:43:57.715ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "8d73e8ee-829b-42e3-a792-e7b5d6e65c95", "content-length": "835", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
68922025-04-30T06:43:57.716ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
68932025-04-30T06:43:57.718ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a95f6e0e-34fa-45aa-871a-0e5f672cfaa8", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
68942025-04-30T06:43:57.719ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
68952025-04-30T06:43:57.721ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0a9acf03-38a1-4d81-9823-a71e33f9f1ab", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
68962025-04-30T06:43:57.722ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
68972025-04-30T06:43:57.726ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c365e74f-42a4-4f91-b7a6-2211dc9b245a", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
68982025-04-30T06:43:57.726ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
68992025-04-30T06:43:57.730ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1ec5e995-c69c-4dca-8a60-69115159f5c4", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
69002025-04-30T06:43:57.730ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
69012025-04-30T06:43:57.732ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "318cf69e-c414-43e4-8c9a-395cde477482", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
69022025-04-30T06:43:57.732ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
69032025-04-30T06:43:57.735ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd1b0eb9-e261-46b6-aa80-3475c29bf446", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
69042025-04-30T06:43:57.735ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
69052025-04-30T06:43:57.738ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8f988177-7c03-4969-b5b3-d3841da871a3", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
69062025-04-30T06:43:57.738ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
69072025-04-30T06:43:57.740ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "11126332-c491-48bf-bf14-50d6bcca5432", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
69082025-04-30T06:43:57.740ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
69092025-04-30T06:43:57.741ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "92d8b192-6917-4731-8a31-34fcdfcdb433", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
69102025-04-30T06:43:57.741ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
69112025-04-30T06:43:57.744ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a147e6d7-bd49-49f3-958a-5d849a069931", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
69122025-04-30T06:43:57.745ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
69132025-04-30T06:43:57.746ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4835fa9a-7ba5-4041-a042-5eb12ccd8e2b", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
69142025-04-30T06:43:57.746ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
69152025-04-30T06:43:57.751ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8fbba3c5-411a-4d9d-bb94-3ee8112c14d2", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
69162025-04-30T06:43:57.751ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
69172025-04-30T06:43:57.754ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ca275460-e286-4500-953d-7d2d08e1b44c", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
69182025-04-30T06:43:57.754ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
69192025-04-30T06:43:57.758ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "441319e7-3d67-44ae-bd92-4b2ca772053d", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
69202025-04-30T06:43:57.758ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
69212025-04-30T06:43:57.760ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e7d28418-81a0-45be-abd6-03ec1c9b3647", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
69222025-04-30T06:43:57.761ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
69232025-04-30T06:43:57.766ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
69242025-04-30T06:43:57.766ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
69252025-04-30T06:43:57.766ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
69262025-04-30T06:43:57.766ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
69272025-04-30T06:43:57.767ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
69282025-04-30T06:43:57.767ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e20602ad-4845-483f-b1c4-8b32f6622462", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
69292025-04-30T06:43:57.767ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
69302025-04-30T06:43:57.770ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69312025-04-30T06:43:57.781ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
69322025-04-30T06:43:57.782ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
69332025-04-30T06:43:57.782ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
69342025-04-30T06:43:57.782ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
69352025-04-30T06:43:57.788ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
69362025-04-30T06:43:57.788ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
69372025-04-30T06:43:57.789ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
69382025-04-30T06:43:57.797ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "12d1fa24-200a-4252-a55b-c2372fad6185", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
69392025-04-30T06:43:57.798ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
69402025-04-30T06:43:57.800ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "50730084-f624-4274-9365-bd275a2ac746", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
69412025-04-30T06:43:57.801ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
69422025-04-30T06:43:57.802ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa5729bf-98e6-4f07-a669-8c4f7eadede8", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
69432025-04-30T06:43:57.802ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
69442025-04-30T06:43:57.803ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "361233bc-a450-457b-9777-56dee5c437d4", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
69452025-04-30T06:43:57.804ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
69462025-04-30T06:43:57.805ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "bc1693a2-efb2-4926-a457-37d9081badf0", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
69472025-04-30T06:43:57.805ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
69482025-04-30T06:43:57.807ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e47ab9f2-1521-4b5c-86a6-377153a87fb5", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
69492025-04-30T06:43:57.807ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
69502025-04-30T06:43:57.810ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0edbf589-e3e6-496d-b882-637d62fc1672", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
69512025-04-30T06:43:57.810ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
69522025-04-30T06:43:57.812ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f4c8b7ea-d292-46d8-b1e9-e2ffb6e22186", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
69532025-04-30T06:43:57.813ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
69542025-04-30T06:43:57.815ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "24785233-791f-405e-ba7f-ea7e7e02f1dc", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
69552025-04-30T06:43:57.815ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
69562025-04-30T06:43:57.818ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f1ab7a0-aeec-42dd-a372-ef09fcc65950", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
69572025-04-30T06:43:57.818ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
69582025-04-30T06:43:57.821ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bc372770-1596-42f9-8906-a0c74cfd4cbe", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
69592025-04-30T06:43:57.821ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
69602025-04-30T06:43:57.825ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
69612025-04-30T06:43:57.828ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5f6de572-25a7-4280-94be-880f574bf412", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
69622025-04-30T06:43:57.829ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
69632025-04-30T06:43:57.831ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6cfc64ea-dd6e-4098-937e-4a13d7b8ec42", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
69642025-04-30T06:43:57.831ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
69652025-04-30T06:43:57.832ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e9e773d4-b768-4018-b6ce-7fcacbc7a38a", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
69662025-04-30T06:43:57.833ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
69672025-04-30T06:43:57.834ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "10138364-75b5-400f-9c63-6f700e9f9d56", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
69682025-04-30T06:43:57.834ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
69692025-04-30T06:43:57.837ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3b52c243-8d64-44f5-99dd-572fba545fc2", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
69702025-04-30T06:43:57.838ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
69712025-04-30T06:43:57.839ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5676d8ab-b5a5-4a4b-8f5a-6ae3cf939aac", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
69722025-04-30T06:43:57.839ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
69732025-04-30T06:43:57.844ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "17253a72-2c59-4a2e-b8bd-f285b25e9c0f", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
69742025-04-30T06:43:57.844ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
69752025-04-30T06:43:57.846ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "290ae969-b0ab-47aa-887a-f5f9fe497430", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
69762025-04-30T06:43:57.848ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
69772025-04-30T06:43:57.856ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5179edbb-e2cc-4429-b9dc-8f4cf9e8f6eb", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
69782025-04-30T06:43:57.857ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
69792025-04-30T06:43:57.859ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d33ee2c5-f49f-4832-9c28-694e37813961", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
69802025-04-30T06:43:57.859ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
69812025-04-30T06:43:57.864ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e67ca14b-656e-43ab-b383-3e36a8163564", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
69822025-04-30T06:43:57.864ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
69832025-04-30T06:43:57.867ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b2d1e26f-15ab-4503-ac02-a0d57acccfcb", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
69842025-04-30T06:43:57.867ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
69852025-04-30T06:43:57.869ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e683d73f-e3c0-4c73-8773-7399ab1f3ea5", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
69862025-04-30T06:43:57.869ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
69872025-04-30T06:43:57.870ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "093b130a-abcb-49f7-bff2-725f3d9679b2", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
69882025-04-30T06:43:57.870ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
69892025-04-30T06:43:57.871ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4af2c879-7214-43e0-91ad-00dc7e19afea", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
69902025-04-30T06:43:57.871ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
69912025-04-30T06:43:57.873ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3204b62a-8737-4abd-880c-8fa5a1c19283", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
69922025-04-30T06:43:57.873ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
69932025-04-30T06:43:57.874ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0e3e3dd3-a265-4819-b8cc-5a24fb030ee5", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
69942025-04-30T06:43:57.874ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
69952025-04-30T06:43:57.874ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
69962025-04-30T06:43:57.874ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
69972025-04-30T06:43:57.874ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
69982025-04-30T06:43:57.875ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
69992025-04-30T06:43:57.876ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
70002025-04-30T06:43:57.876ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
70012025-04-30T06:43:57.876ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
70022025-04-30T06:43:57.876ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
70032025-04-30T06:43:57.878ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
70042025-04-30T06:43:57.879ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70052025-04-30T06:43:57.890ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
70062025-04-30T06:43:57.890ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
70072025-04-30T06:43:57.890ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
70082025-04-30T06:43:57.891ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
70092025-04-30T06:43:57.891ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70102025-04-30T06:43:57.894ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
70112025-04-30T06:43:57.895ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
70122025-04-30T06:43:57.895ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
70132025-04-30T06:43:57.895ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
70142025-04-30T06:43:57.895ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
70152025-04-30T06:43:57.896ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
70162025-04-30T06:43:57.899ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
70172025-04-30T06:43:57.899ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
70182025-04-30T06:43:57.900ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
70192025-04-30T06:43:57.900ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70202025-04-30T06:43:57.904ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
70212025-04-30T06:43:57.904ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
70222025-04-30T06:43:57.904ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
70232025-04-30T06:43:57.905ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
70242025-04-30T06:43:57.965ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
70252025-04-30T06:43:57.965ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory
70262025-04-30T06:43:57.966ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:39426 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
70272025-04-30T06:43:57.966ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:39426 req_id = a9680a66-a840-4c52-9cff-e0664e18d165 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
70282025-04-30T06:43:57.967ZINFOtest_unauthorized (dropshot): request completed latency_us = 748 local_addr = [::1]:57786 method = GET remote_addr = [::1]:39426 req_id = a9680a66-a840-4c52-9cff-e0664e18d165 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
70292025-04-30T06:43:57.968ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = http://[::1]:57786 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:57786/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "a9680a66-a840-4c52-9cff-e0664e18d165", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
70302025-04-30T06:43:57.970ZDEBGtest_unauthorized (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
70312025-04-30T06:43:57.970ZDEBGtest_unauthorized (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
70322025-04-30T06:43:57.970ZDEBGtest_unauthorized (BackgroundTasks): finished collection background_task = inventory_collection
70332025-04-30T06:43:57.972ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
70342025-04-30T06:43:57.972ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
70352025-04-30T06:43:57.972ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
70362025-04-30T06:43:57.972ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
70372025-04-30T06:43:57.973ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
70382025-04-30T06:43:57.978ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70392025-04-30T06:43:57.983ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
70402025-04-30T06:43:57.983ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
70412025-04-30T06:43:57.983ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
70422025-04-30T06:43:57.984ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
70432025-04-30T06:43:58.019ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
70442025-04-30T06:43:58.029ZWARNtest_unauthorized (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
70452025-04-30T06:43:58.029ZWARNtest_unauthorized (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
70462025-04-30T06:43:58.029ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 405.490453ms iteration = 21 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
70472025-04-30T06:43:58.029ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 22 reason = Signaled
70482025-04-30T06:43:58.029ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
70492025-04-30T06:43:58.029ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
70502025-04-30T06:43:58.029ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
70512025-04-30T06:43:58.029ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
70522025-04-30T06:43:58.030ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
70532025-04-30T06:43:58.032ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70542025-04-30T06:43:58.036ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
70552025-04-30T06:43:58.036ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
70562025-04-30T06:43:58.036ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
70572025-04-30T06:43:58.036ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
70582025-04-30T06:43:58.045ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1a8b2318-abf8-406d-a389-112ab91a0de8, 0)]
70592025-04-30T06:43:58.045ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
70602025-04-30T06:43:58.046ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 68 a7 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 h............_mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:64045 req_id = 40ed4347-8d2d-4283-92f6-34a43b73ff28
70612025-04-30T06:43:58.046ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 26791,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64045 req_id = 40ed4347-8d2d-4283-92f6-34a43b73ff28
70622025-04-30T06:43:58.048ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
70632025-04-30T06:43:58.048ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
70642025-04-30T06:43:58.048ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:64045 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 40ed4347-8d2d-4283-92f6-34a43b73ff28
70652025-04-30T06:43:58.048ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4302, tv_nsec: 847662265 } })
70662025-04-30T06:43:58.048ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
70672025-04-30T06:43:58.049ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: af 93 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:63724 req_id = e2f511e3-b9ad-47e9-bd65-0a040ba08e0a
70682025-04-30T06:43:58.049ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 44947,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63724 req_id = e2f511e3-b9ad-47e9-bd65-0a040ba08e0a
70692025-04-30T06:43:58.049ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
70702025-04-30T06:43:58.049ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:63724 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = e2f511e3-b9ad-47e9-bd65-0a040ba08e0a
70712025-04-30T06:43:58.098ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
70722025-04-30T06:43:58.099ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
70732025-04-30T06:43:58.099ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
70742025-04-30T06:43:58.099ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
70752025-04-30T06:43:58.099ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
70762025-04-30T06:43:58.100ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
70772025-04-30T06:43:58.100ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 0e 8e 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:50051 req_id = 29fa9a1a-d809-4d5f-8b3e-dcfd0b24e554
70782025-04-30T06:43:58.100ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 3726,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:50051 req_id = 29fa9a1a-d809-4d5f-8b3e-dcfd0b24e554
70792025-04-30T06:43:58.101ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
70802025-04-30T06:43:58.101ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 3726, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:50051 req_id = 29fa9a1a-d809-4d5f-8b3e-dcfd0b24e554
70812025-04-30T06:43:58.101ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
70822025-04-30T06:43:58.101ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
70832025-04-30T06:43:58.101ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
70842025-04-30T06:43:58.103ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "e4cbe204-6394-45e6-a1ec-91f92252acae", "content-length": "835", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
70852025-04-30T06:43:58.104ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
70862025-04-30T06:43:58.105ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70872025-04-30T06:43:58.111ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
70882025-04-30T06:43:58.111ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
70892025-04-30T06:43:58.111ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
70902025-04-30T06:43:58.111ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
70912025-04-30T06:43:58.114ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "72e3bfb2-8c8e-4f7f-bfe3-5fda7d62fcdf", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
70922025-04-30T06:43:58.114ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
70932025-04-30T06:43:58.115ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
70942025-04-30T06:43:58.115ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
70952025-04-30T06:43:58.115ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
70962025-04-30T06:43:58.119ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "27956803-ee32-4336-a18d-7f55504339ce", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
70972025-04-30T06:43:58.119ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
70982025-04-30T06:43:58.122ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "342ea05e-1b20-4b1e-9119-136a96ff51c1", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
70992025-04-30T06:43:58.122ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
71002025-04-30T06:43:58.124ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b51b2053-6423-43c0-8aff-4a522e59316e", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
71012025-04-30T06:43:58.124ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
71022025-04-30T06:43:58.127ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "058012f4-7a59-4270-b807-1ea12caa201f", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
71032025-04-30T06:43:58.127ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
71042025-04-30T06:43:58.129ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9447e33c-e165-411f-90b2-53773c27c053", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
71052025-04-30T06:43:58.129ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
71062025-04-30T06:43:58.131ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b1e7cecd-01d6-4b8c-9f3f-59deb9c5308f", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
71072025-04-30T06:43:58.131ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
71082025-04-30T06:43:58.132ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a4a4cc0d-85f9-4efd-a017-0562eeb08362", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
71092025-04-30T06:43:58.132ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
71102025-04-30T06:43:58.137ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5a6cc9a2-0be5-4669-9a28-6805e98b9e0a", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
71112025-04-30T06:43:58.137ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
71122025-04-30T06:43:58.140ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1862bdd8-7c06-4b06-a761-ff189a107df1", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
71132025-04-30T06:43:58.140ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
71142025-04-30T06:43:58.141ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8e02e8be-c4a9-449e-a433-3ac68ffee3cb", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
71152025-04-30T06:43:58.142ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
71162025-04-30T06:43:58.146ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "13a53d16-950e-483d-9540-5e6de2f72c76", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
71172025-04-30T06:43:58.146ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
71182025-04-30T06:43:58.149ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
71192025-04-30T06:43:58.152ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "702c6842-2b10-4daf-a5f2-0ba5da72aeac", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
71202025-04-30T06:43:58.153ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
71212025-04-30T06:43:58.158ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "28e54476-39a6-4ecb-a5c0-fc2856453263", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
71222025-04-30T06:43:58.158ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
71232025-04-30T06:43:58.162ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9887958a-acc4-4ccc-af85-a6452944ba04", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
71242025-04-30T06:43:58.162ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
71252025-04-30T06:43:58.166ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e7bd4114-13c5-48db-a694-f6bf2f9c36b3", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
71262025-04-30T06:43:58.166ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
71272025-04-30T06:43:58.169ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1815e62b-b925-4df6-a917-7924c78d67d8", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
71282025-04-30T06:43:58.170ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
71292025-04-30T06:43:58.173ZDEBGtest_unauthorized (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
71302025-04-30T06:43:58.173ZDEBGtest_unauthorized (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
71312025-04-30T06:43:58.174ZDEBGtest_unauthorized (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a
71322025-04-30T06:43:58.175ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
71332025-04-30T06:43:58.180ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 2 table_name = fields_string
71342025-04-30T06:43:58.180ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
71352025-04-30T06:43:58.180ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7c4cba23-2202-4a90-8b06-2e7396717b12", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
71362025-04-30T06:43:58.180ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
71372025-04-30T06:43:58.182ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c931ef8d-92d8-4870-8701-5d2470f7810f", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
71382025-04-30T06:43:58.182ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
71392025-04-30T06:43:58.182ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 1 table_name = measurements_i64
71402025-04-30T06:43:58.182ZTRACtest_unauthorized (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
71412025-04-30T06:43:58.184ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "41c4ba17-e0ec-4204-aca5-8b6c70d56f0e", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
71422025-04-30T06:43:58.184ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
71432025-04-30T06:43:58.185ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "775b2d20-eb5e-49dd-85a5-58c3cf4b292e", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
71442025-04-30T06:43:58.185ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
71452025-04-30T06:43:58.186ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7723e683-2a61-4c5f-8cf7-39f3e881af58", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
71462025-04-30T06:43:58.186ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
71472025-04-30T06:43:58.199ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ea9ef86b-9d34-48d1-9a38-1866d9e4d43d", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
71482025-04-30T06:43:58.199ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
71492025-04-30T06:43:58.200ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
71502025-04-30T06:43:58.200ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
71512025-04-30T06:43:58.200ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
71522025-04-30T06:43:58.200ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
71532025-04-30T06:43:58.200ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
71542025-04-30T06:43:58.201ZDEBGtest_unauthorized (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
71552025-04-30T06:43:58.201ZDEBGtest_unauthorized (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
71562025-04-30T06:43:58.204ZTRACtest_unauthorized (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
71572025-04-30T06:43:58.206ZDEBGtest_unauthorized (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
71582025-04-30T06:43:58.206ZTRACtest_unauthorized (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
71592025-04-30T06:43:58.206ZTRACtest_unauthorized (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
71602025-04-30T06:43:58.209ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7701da40-dd6e-42f1-8780-50cb300703b8", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
71612025-04-30T06:43:58.209ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
71622025-04-30T06:43:58.210ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71632025-04-30T06:43:58.216ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
71642025-04-30T06:43:58.217ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
71652025-04-30T06:43:58.217ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
71662025-04-30T06:43:58.218ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
71672025-04-30T06:43:58.234ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
71682025-04-30T06:43:58.234ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
71692025-04-30T06:43:58.234ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
71702025-04-30T06:43:58.238ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "62ab9afe-30b3-40db-9a3f-570fa66dc7ff", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
71712025-04-30T06:43:58.238ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
71722025-04-30T06:43:58.241ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "21eeaca7-6de6-420d-9686-574748d0c4e9", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
71732025-04-30T06:43:58.242ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
71742025-04-30T06:43:58.245ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2ae0816c-6d99-405b-8713-b8c289fbf336", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
71752025-04-30T06:43:58.245ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
71762025-04-30T06:43:58.250ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "93d1fcce-9419-4fae-956c-e633f2578b6c", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
71772025-04-30T06:43:58.250ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
71782025-04-30T06:43:58.275ZWARNtest_unauthorized (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
71792025-04-30T06:43:58.294ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "935c2f9d-00b5-4f71-8340-36a85836a333", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
71802025-04-30T06:43:58.295ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
71812025-04-30T06:43:58.299ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "70bae1be-ade2-409b-a93d-5612fcd45da3", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
71822025-04-30T06:43:58.299ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
71832025-04-30T06:43:58.302ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e3948bf9-516d-41e0-9f7f-2c0020a980a0", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
71842025-04-30T06:43:58.302ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
71852025-04-30T06:43:58.304ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8cb6d744-cf75-4c63-8767-54cdf35db82a", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
71862025-04-30T06:43:58.304ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
71872025-04-30T06:43:58.313ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "281a4a04-3508-42ef-ad94-7f779e7253f3", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
71882025-04-30T06:43:58.314ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
71892025-04-30T06:43:58.318ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
71902025-04-30T06:43:58.321ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a86fae0-6ed6-4e49-bfcc-e0998f782972", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
71912025-04-30T06:43:58.321ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
71922025-04-30T06:43:58.324ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2d79cd50-749c-4278-bbdf-eab8332bbfa3", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
71932025-04-30T06:43:58.324ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
71942025-04-30T06:43:58.350ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "56256962-7bad-4450-80ff-02628c9630c4", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
71952025-04-30T06:43:58.351ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
71962025-04-30T06:43:58.353ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5e9f0124-8528-4745-8907-6b97fc4e0902", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
71972025-04-30T06:43:58.353ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
71982025-04-30T06:43:58.355ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "76635aef-dd6d-4084-9166-5122f516411c", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
71992025-04-30T06:43:58.355ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
72002025-04-30T06:43:58.363ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7e1d6147-560a-4ee3-9995-63bd1834b7a4", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
72012025-04-30T06:43:58.363ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
72022025-04-30T06:43:58.367ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a2a531a5-e516-4383-a97a-f46a2b232e03", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
72032025-04-30T06:43:58.367ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
72042025-04-30T06:43:58.368ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b214391b-73f3-4b0a-9e82-3f0f7cadb1b3", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
72052025-04-30T06:43:58.368ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
72062025-04-30T06:43:58.369ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
72072025-04-30T06:43:58.369ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
72082025-04-30T06:43:58.369ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
72092025-04-30T06:43:58.369ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
72102025-04-30T06:43:58.370ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
72112025-04-30T06:43:58.370ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d6d63f3b-530d-42fa-8359-5b523dd53868", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
72122025-04-30T06:43:58.371ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
72132025-04-30T06:43:58.372ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6a49443c-f7ef-498e-b68d-af776a119776", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
72142025-04-30T06:43:58.372ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
72152025-04-30T06:43:58.372ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72162025-04-30T06:43:58.417ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
72172025-04-30T06:43:58.417ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
72182025-04-30T06:43:58.417ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
72192025-04-30T06:43:58.417ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
72202025-04-30T06:43:58.422ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "202d6f3f-f691-48c1-9cbb-839b85d56c66", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
72212025-04-30T06:43:58.422ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
72222025-04-30T06:43:58.422ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
72232025-04-30T06:43:58.422ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
72242025-04-30T06:43:58.422ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
72252025-04-30T06:43:58.423ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
72262025-04-30T06:43:58.424ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
72272025-04-30T06:43:58.424ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
72282025-04-30T06:43:58.425ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
72292025-04-30T06:43:58.429ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72302025-04-30T06:43:58.433ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
72312025-04-30T06:43:58.433ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
72322025-04-30T06:43:58.433ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
72332025-04-30T06:43:58.433ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
72342025-04-30T06:43:58.433ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
72352025-04-30T06:43:58.434ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
72362025-04-30T06:43:58.445ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72372025-04-30T06:43:58.448ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
72382025-04-30T06:43:58.448ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
72392025-04-30T06:43:58.448ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
72402025-04-30T06:43:58.451ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
72412025-04-30T06:43:58.633ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
72422025-04-30T06:43:58.633ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory
72432025-04-30T06:43:58.634ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:59249 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
72442025-04-30T06:43:58.635ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:59249 req_id = 9a42cce2-7825-4ef9-a8f1-d3b5bcac21e2 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
72452025-04-30T06:43:58.636ZINFOtest_unauthorized (dropshot): request completed latency_us = 805 local_addr = [::1]:57786 method = GET remote_addr = [::1]:59249 req_id = 9a42cce2-7825-4ef9-a8f1-d3b5bcac21e2 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
72462025-04-30T06:43:58.637ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = http://[::1]:57786 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:57786/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "9a42cce2-7825-4ef9-a8f1-d3b5bcac21e2", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
72472025-04-30T06:43:58.641ZDEBGtest_unauthorized (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
72482025-04-30T06:43:58.641ZDEBGtest_unauthorized (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
72492025-04-30T06:43:58.641ZDEBGtest_unauthorized (BackgroundTasks): finished collection background_task = inventory_collection
72502025-04-30T06:43:58.644ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
72512025-04-30T06:43:58.644ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
72522025-04-30T06:43:58.644ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
72532025-04-30T06:43:58.644ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
72542025-04-30T06:43:58.645ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
72552025-04-30T06:43:58.648ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72562025-04-30T06:43:58.661ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
72572025-04-30T06:43:58.661ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
72582025-04-30T06:43:58.662ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
72592025-04-30T06:43:58.662ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
72602025-04-30T06:43:58.883ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
72612025-04-30T06:43:58.915ZWARNtest_unauthorized (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
72622025-04-30T06:43:58.916ZWARNtest_unauthorized (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
72632025-04-30T06:43:58.916ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 887.185229ms iteration = 22 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
72642025-04-30T06:43:58.916ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 23 reason = Signaled
72652025-04-30T06:43:58.917ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
72662025-04-30T06:43:58.917ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
72672025-04-30T06:43:58.917ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
72682025-04-30T06:43:58.917ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
72692025-04-30T06:43:58.918ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
72702025-04-30T06:43:58.949ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
72712025-04-30T06:43:58.949ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
72722025-04-30T06:43:58.949ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
72732025-04-30T06:43:58.949ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
72742025-04-30T06:43:58.950ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
72752025-04-30T06:43:58.951ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72762025-04-30T06:43:58.957ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
72772025-04-30T06:43:58.958ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
72782025-04-30T06:43:58.958ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
72792025-04-30T06:43:58.958ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
72802025-04-30T06:43:58.960ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72812025-04-30T06:43:58.966ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
72822025-04-30T06:43:58.966ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
72832025-04-30T06:43:58.966ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
72842025-04-30T06:43:58.967ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
72852025-04-30T06:43:58.978ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1a8b2318-abf8-406d-a389-112ab91a0de8, 0)]
72862025-04-30T06:43:58.978ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
72872025-04-30T06:43:58.979ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: c0 60 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .`..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:64724 req_id = d9283b74-4b2b-436d-af95-f744583228c8
72882025-04-30T06:43:58.979ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 49248,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64724 req_id = d9283b74-4b2b-436d-af95-f744583228c8
72892025-04-30T06:43:58.979ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
72902025-04-30T06:43:58.980ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
72912025-04-30T06:43:58.980ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:64724 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = d9283b74-4b2b-436d-af95-f744583228c8
72922025-04-30T06:43:58.980ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
72932025-04-30T06:43:58.980ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
72942025-04-30T06:43:58.981ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
72952025-04-30T06:43:58.982ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4303, tv_nsec: 781226180 } })
72962025-04-30T06:43:58.982ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
72972025-04-30T06:43:58.983ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 1d 16 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:57795 req_id = 48de2a57-d874-4852-a788-732ccdb4704f
72982025-04-30T06:43:58.983ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 7446,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:57795 req_id = 48de2a57-d874-4852-a788-732ccdb4704f
72992025-04-30T06:43:58.983ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
73002025-04-30T06:43:58.983ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:57795 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 48de2a57-d874-4852-a788-732ccdb4704f
73012025-04-30T06:43:59.030ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
73022025-04-30T06:43:59.032ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: ef 30 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c .0..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:53616 req_id = e7db2738-c5fe-4bcd-94ff-9508988fd28f
73032025-04-30T06:43:59.032ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 61232,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:53616 req_id = e7db2738-c5fe-4bcd-94ff-9508988fd28f
73042025-04-30T06:43:59.032ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
73052025-04-30T06:43:59.032ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 61232, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:53616 req_id = e7db2738-c5fe-4bcd-94ff-9508988fd28f
73062025-04-30T06:43:59.032ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
73072025-04-30T06:43:59.032ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
73082025-04-30T06:43:59.032ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
73092025-04-30T06:43:59.034ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "225190f0-c2f0-4f14-9666-d61241e2abda", "content-length": "835", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
73102025-04-30T06:43:59.035ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
73112025-04-30T06:43:59.037ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ce20b306-9519-4f6d-a03c-51267d2b8f57", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
73122025-04-30T06:43:59.037ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
73132025-04-30T06:43:59.039ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3dcc249d-c6c0-4820-b908-cb73a4d10ff7", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
73142025-04-30T06:43:59.039ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
73152025-04-30T06:43:59.043ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2e3f7664-b9ad-4cec-93fb-63c4e4f4f48a", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
73162025-04-30T06:43:59.043ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
73172025-04-30T06:43:59.046ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a30234fe-fbb2-4362-880b-2b11fb116431", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
73182025-04-30T06:43:59.046ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
73192025-04-30T06:43:59.048ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "439e6d41-a531-423f-b708-ae9bd9c38740", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
73202025-04-30T06:43:59.048ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
73212025-04-30T06:43:59.050ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d880a5d8-ecc6-4534-a323-cf3593b79d04", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
73222025-04-30T06:43:59.050ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
73232025-04-30T06:43:59.052ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1326fb9f-22cf-4fdf-a5bb-68dcdff03dae", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
73242025-04-30T06:43:59.053ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
73252025-04-30T06:43:59.055ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "975d7d68-4545-4996-b1f6-ca1f0f486809", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
73262025-04-30T06:43:59.055ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
73272025-04-30T06:43:59.060ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ce148332-7ba9-41e4-abec-7e2bed15bb1c", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
73282025-04-30T06:43:59.061ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
73292025-04-30T06:43:59.062ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
73302025-04-30T06:43:59.065ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f8abfea1-2c4c-486f-82a7-ea0319733029", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
73312025-04-30T06:43:59.065ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
73322025-04-30T06:43:59.068ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d1ec39a7-b137-4fe1-8027-c8a7f1084efd", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
73332025-04-30T06:43:59.068ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
73342025-04-30T06:43:59.069ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c7c78d0e-ef34-4dd6-8f77-7b17f96a46cf", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
73352025-04-30T06:43:59.069ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
73362025-04-30T06:43:59.072ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "05a450ab-085f-443f-b8e6-db49c3050fb6", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
73372025-04-30T06:43:59.072ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
73382025-04-30T06:43:59.074ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ef554608-4ab4-412c-9214-43565d7f5943", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
73392025-04-30T06:43:59.074ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
73402025-04-30T06:43:59.077ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "62fa9b4c-98a3-46ff-a338-be3b25318fcf", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
73412025-04-30T06:43:59.077ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
73422025-04-30T06:43:59.079ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2c1d667a-1599-4cd3-8635-95f12286a697", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
73432025-04-30T06:43:59.079ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
73442025-04-30T06:43:59.082ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1e4c1015-fa8e-450f-b1bb-806ac80efcbe", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
73452025-04-30T06:43:59.082ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
73462025-04-30T06:43:59.098ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f43d89cd-2fcb-4ff0-9ef3-ad96be3f146d", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
73472025-04-30T06:43:59.098ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
73482025-04-30T06:43:59.099ZINFOtest_unauthorized (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
73492025-04-30T06:43:59.099ZDEBGtest_unauthorized (oximeter): client request body = None method = GET uri = http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
73502025-04-30T06:43:59.100ZTRACtest_unauthorized (dropshot_internal): incoming request local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 9e11ee77-7ff5-4f35-a80a-128d95b9ff2d uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
73512025-04-30T06:43:59.100ZTRACtest_unauthorized (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 9e11ee77-7ff5-4f35-a80a-128d95b9ff2d resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
73522025-04-30T06:43:59.100ZDEBGtest_unauthorized (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 9e11ee77-7ff5-4f35-a80a-128d95b9ff2d roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
73532025-04-30T06:43:59.101ZDEBGtest_unauthorized (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 9e11ee77-7ff5-4f35-a80a-128d95b9ff2d resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
73542025-04-30T06:43:59.101ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ad9a9ac9-f8d6-46a4-be1f-0bf0f528f977", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
73552025-04-30T06:43:59.101ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
73562025-04-30T06:43:59.103ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c068bea5-9fbf-404b-8466-3507c1078ec8", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
73572025-04-30T06:43:59.103ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
73582025-04-30T06:43:59.104ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "afc96eca-d744-4bf6-8f3d-bf6ae87cec4f", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
73592025-04-30T06:43:59.104ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
73602025-04-30T06:43:59.104ZINFOtest_unauthorized (dropshot_internal): request completed latency_us = 4864 local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 9e11ee77-7ff5-4f35-a80a-128d95b9ff2d response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
73612025-04-30T06:43:59.105ZDEBGtest_unauthorized (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "9e11ee77-7ff5-4f35-a80a-128d95b9ff2d", "content-length": "405", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
73622025-04-30T06:43:59.105ZDEBGtest_unauthorized (oximeter): client request body = None method = GET uri = http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
73632025-04-30T06:43:59.106ZTRACtest_unauthorized (dropshot_internal): incoming request local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = e5e514c5-ff6f-4e4e-a46a-472fa19ee2f3 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
73642025-04-30T06:43:59.106ZTRACtest_unauthorized (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = e5e514c5-ff6f-4e4e-a46a-472fa19ee2f3 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
73652025-04-30T06:43:59.106ZDEBGtest_unauthorized (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = e5e514c5-ff6f-4e4e-a46a-472fa19ee2f3 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
73662025-04-30T06:43:59.106ZDEBGtest_unauthorized (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = e5e514c5-ff6f-4e4e-a46a-472fa19ee2f3 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
73672025-04-30T06:43:59.107ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e64e38ee-1b31-4d0d-b173-b58e4f4f2585", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
73682025-04-30T06:43:59.107ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
73692025-04-30T06:43:59.108ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "49cce5e4-bff9-44bf-a2e0-5699c433b74d", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
73702025-04-30T06:43:59.108ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
73712025-04-30T06:43:59.109ZINFOtest_unauthorized (dropshot_internal): request completed latency_us = 3667 local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = e5e514c5-ff6f-4e4e-a46a-472fa19ee2f3 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
73722025-04-30T06:43:59.110ZDEBGtest_unauthorized (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "e5e514c5-ff6f-4e4e-a46a-472fa19ee2f3", "content-length": "29", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
73732025-04-30T06:43:59.110ZTRACtest_unauthorized (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 8d752adb-08b9-405f-a11b-66019bf447b5
73742025-04-30T06:43:59.110ZTRACtest_unauthorized (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
73752025-04-30T06:43:59.110ZINFOtest_unauthorized (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
73762025-04-30T06:43:59.117ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
73772025-04-30T06:43:59.117ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
73782025-04-30T06:43:59.117ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
73792025-04-30T06:43:59.117ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
73802025-04-30T06:43:59.118ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
73812025-04-30T06:43:59.118ZDEBGtest_unauthorized (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 8d752adb-08b9-405f-a11b-66019bf447b5
73822025-04-30T06:43:59.119ZDEBGtest_unauthorized (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 8d752adb-08b9-405f-a11b-66019bf447b5
73832025-04-30T06:43:59.120ZINFOtest_unauthorized (dropshot): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:1025 local_addr = 127.0.0.1:53884 remote_addr = 127.0.0.1:47166
73842025-04-30T06:43:59.121ZTRACtest_unauthorized (dropshot): incoming request local_addr = 127.0.0.1:53884 method = GET remote_addr = 127.0.0.1:47166 req_id = be1cb519-2131-4347-83a6-564a7c4f1ce9 uri = /8d752adb-08b9-405f-a11b-66019bf447b5
73852025-04-30T06:43:59.126ZINFOtest_unauthorized (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 5550 local_addr = 127.0.0.1:53884 method = GET remote_addr = 127.0.0.1:47166 req_id = be1cb519-2131-4347-83a6-564a7c4f1ce9 response_code = 200 uri = /8d752adb-08b9-405f-a11b-66019bf447b5
73862025-04-30T06:43:59.126ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73872025-04-30T06:43:59.132ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
73882025-04-30T06:43:59.132ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
73892025-04-30T06:43:59.132ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
73902025-04-30T06:43:59.133ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
73912025-04-30T06:43:59.140ZTRACtest_unauthorized (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 8d752adb-08b9-405f-a11b-66019bf447b5
73922025-04-30T06:43:59.154ZDEBGtest_unauthorized (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 5 producer_id = 8d752adb-08b9-405f-a11b-66019bf447b5
73932025-04-30T06:43:59.154ZTRACtest_unauthorized (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 8d752adb-08b9-405f-a11b-66019bf447b5
73942025-04-30T06:43:59.154ZTRACtest_unauthorized (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 8d752adb-08b9-405f-a11b-66019bf447b5
73952025-04-30T06:43:59.155ZDEBGtest_unauthorized (results-sink): inserting 18 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
73962025-04-30T06:43:59.155ZDEBGtest_unauthorized (results-sink-cluster): inserting 18 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
73972025-04-30T06:43:59.155ZDEBGtest_unauthorized (clickhouse-client): unrolling 18 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a
73982025-04-30T06:43:59.157ZDEBGtest_unauthorized (clickhouse-client): retrieving timeseries schema from database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a
73992025-04-30T06:43:59.157ZTRACtest_unauthorized (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a sql = SELECT * FROM oximeter.timeseries_schema WHERE timeseries_name NOT IN ('integration_target:integration_metric') FORMAT Native;
74002025-04-30T06:43:59.160ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
74012025-04-30T06:43:59.160ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
74022025-04-30T06:43:59.161ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
74032025-04-30T06:43:59.162ZTRACtest_unauthorized (clickhouse-client): no new timeseries schema in database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a
74042025-04-30T06:43:59.162ZDEBGtest_unauthorized (clickhouse-client): new timeseries schema collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a schema = TimeseriesSchema { timeseries_name: TimeseriesName("http_service:request_latency_histogram"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "name", field_type: String, source: Target, description: "" }, FieldSchema { name: "operation_id", field_type: String, source: Metric, description: "" }, FieldSchema { name: "status_code", field_type: U16, source: Metric, description: "" }}, datum_type: HistogramU64, version: 1, authz_scope: Fleet, units: Count, created: 2025-04-30T06:43:59.157045039Z } timeseries_name = http_service:request_latency_histogram
74052025-04-30T06:43:59.166ZDEBGtest_unauthorized (clickhouse-client): retrieving timeseries schema from database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a
74062025-04-30T06:43:59.166ZTRACtest_unauthorized (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a sql = SELECT * FROM oximeter.timeseries_schema WHERE timeseries_name NOT IN ('http_service:request_latency_histogram', 'integration_target:integration_metric') FORMAT Native;
74072025-04-30T06:43:59.167ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "41498ebe-843f-407a-af73-2a2ecc1c5f8a", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
74082025-04-30T06:43:59.167ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
74092025-04-30T06:43:59.169ZTRACtest_unauthorized (clickhouse-client): no new timeseries schema in database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a
74102025-04-30T06:43:59.170ZDEBGtest_unauthorized (clickhouse-client): new timeseries schema collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a schema = TimeseriesSchema { timeseries_name: TimeseriesName("collection_target:cpus_provisioned"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "id", field_type: Uuid, source: Target, description: "" }}, datum_type: I64, version: 1, authz_scope: Fleet, units: Count, created: 2025-04-30T06:43:59.166265871Z } timeseries_name = collection_target:cpus_provisioned
74112025-04-30T06:43:59.171ZDEBGtest_unauthorized (clickhouse-client): retrieving timeseries schema from database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a
74122025-04-30T06:43:59.171ZTRACtest_unauthorized (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a sql = SELECT * FROM oximeter.timeseries_schema WHERE timeseries_name NOT IN ('collection_target:cpus_provisioned', 'http_service:request_latency_histogram', 'integration_target:integration_metric') FORMAT Native;
74132025-04-30T06:43:59.173ZTRACtest_unauthorized (clickhouse-client): no new timeseries schema in database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a
74142025-04-30T06:43:59.173ZDEBGtest_unauthorized (clickhouse-client): new timeseries schema collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a schema = TimeseriesSchema { timeseries_name: TimeseriesName("collection_target:ram_provisioned"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "id", field_type: Uuid, source: Target, description: "" }}, datum_type: I64, version: 1, authz_scope: Fleet, units: Count, created: 2025-04-30T06:43:59.170975926Z } timeseries_name = collection_target:ram_provisioned
74152025-04-30T06:43:59.174ZDEBGtest_unauthorized (clickhouse-client): retrieving timeseries schema from database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a
74162025-04-30T06:43:59.174ZTRACtest_unauthorized (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a sql = SELECT * FROM oximeter.timeseries_schema WHERE timeseries_name NOT IN ('collection_target:cpus_provisioned', 'collection_target:ram_provisioned', 'http_service:request_latency_histogram', 'integration_target:integration_metric') FORMAT Native;
74172025-04-30T06:43:59.175ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1e817242-ed62-4f87-829d-eaa2bd382b35", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
74182025-04-30T06:43:59.177ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
74192025-04-30T06:43:59.177ZTRACtest_unauthorized (clickhouse-client): no new timeseries schema in database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a
74202025-04-30T06:43:59.178ZDEBGtest_unauthorized (clickhouse-client): new timeseries schema collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a schema = TimeseriesSchema { timeseries_name: TimeseriesName("collection_target:virtual_disk_space_provisioned"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "id", field_type: Uuid, source: Target, description: "" }}, datum_type: I64, version: 1, authz_scope: Fleet, units: Count, created: 2025-04-30T06:43:59.174630658Z } timeseries_name = collection_target:virtual_disk_space_provisioned
74212025-04-30T06:43:59.185ZDEBGtest_unauthorized (clickhouse-client): inserting 4 new timeseries schema collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a
74222025-04-30T06:43:59.185ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 6 n_rows = 4 sql = INSERT INTO oximeter.timeseries_schema FORMAT Native
74232025-04-30T06:43:59.194ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 12 sql = INSERT INTO oximeter.fields_string FORMAT Native
74242025-04-30T06:43:59.196ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "861033cc-392f-48a9-81b8-64c3d676a0c4", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
74252025-04-30T06:43:59.197ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
74262025-04-30T06:43:59.200ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 12 table_name = fields_string
74272025-04-30T06:43:59.200ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 5 sql = INSERT INTO oximeter.fields_u16 FORMAT Native
74282025-04-30T06:43:59.200ZDEBGtest_unauthorized (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
74292025-04-30T06:43:59.200ZDEBGtest_unauthorized (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
74302025-04-30T06:43:59.203ZTRACtest_unauthorized (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
74312025-04-30T06:43:59.204ZDEBGtest_unauthorized (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
74322025-04-30T06:43:59.204ZTRACtest_unauthorized (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
74332025-04-30T06:43:59.204ZTRACtest_unauthorized (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
74342025-04-30T06:43:59.205ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c5dca59d-4b8e-4c20-a98a-2257084747c8", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
74352025-04-30T06:43:59.205ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
74362025-04-30T06:43:59.207ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 5 table_name = fields_u16
74372025-04-30T06:43:59.207ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 17 sql = INSERT INTO oximeter.fields_uuid FORMAT Native
74382025-04-30T06:43:59.241ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 17 table_name = fields_uuid
74392025-04-30T06:43:59.242ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 19 n_rows = 5 sql = INSERT INTO oximeter.measurements_histogramu64 FORMAT Native
74402025-04-30T06:43:59.245ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
74412025-04-30T06:43:59.245ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f1d1e76c-54f0-49ed-974e-db37cfcfbc9e", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
74422025-04-30T06:43:59.246ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
74432025-04-30T06:43:59.247ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 5 table_name = measurements_histogramu64
74442025-04-30T06:43:59.247ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 13 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
74452025-04-30T06:43:59.249ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f776ac38-1760-4049-b305-0a6079d566ae", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
74462025-04-30T06:43:59.249ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
74472025-04-30T06:43:59.250ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 13 table_name = measurements_i64
74482025-04-30T06:43:59.250ZTRACtest_unauthorized (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
74492025-04-30T06:43:59.250ZDEBGtest_unauthorized (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
74502025-04-30T06:43:59.251ZDEBGtest_unauthorized (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a
74512025-04-30T06:43:59.251ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
74522025-04-30T06:43:59.252ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d3ca815d-5f09-49e9-84d5-b00011df9bec", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
74532025-04-30T06:43:59.252ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
74542025-04-30T06:43:59.254ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e30768ba-1b56-4203-aa77-7da5cb095bb3", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
74552025-04-30T06:43:59.254ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
74562025-04-30T06:43:59.255ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "29d5f855-50d7-4137-a12f-ed573ac3a644", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
74572025-04-30T06:43:59.255ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
74582025-04-30T06:43:59.255ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 2 table_name = fields_string
74592025-04-30T06:43:59.256ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
74602025-04-30T06:43:59.257ZWARNtest_unauthorized (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
74612025-04-30T06:43:59.257ZTRACtest_unauthorized (results-sink-cluster): batch interval expired, but no samples to insert collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
74622025-04-30T06:43:59.259ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "df26b865-f683-41df-aa04-fa2931407a84", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
74632025-04-30T06:43:59.259ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
74642025-04-30T06:43:59.260ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e73a1947-48fe-43f7-bfcb-e0a5fbee6ff9", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
74652025-04-30T06:43:59.261ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
74662025-04-30T06:43:59.268ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 1 table_name = measurements_i64
74672025-04-30T06:43:59.268ZTRACtest_unauthorized (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
74682025-04-30T06:43:59.269ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ceee74ef-6869-45c6-b783-aff1d29c788b", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
74692025-04-30T06:43:59.269ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
74702025-04-30T06:43:59.271ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6cc45eaa-8256-4edc-98d1-dd940e42d62c", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
74712025-04-30T06:43:59.272ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
74722025-04-30T06:43:59.277ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "418efe35-a502-4036-91f7-e4074f6a89db", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
74732025-04-30T06:43:59.277ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
74742025-04-30T06:43:59.279ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "66085245-651d-4500-a18f-05efb4cc9aff", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
74752025-04-30T06:43:59.279ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
74762025-04-30T06:43:59.281ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0823684e-50cc-4194-8d46-1192168ba0f6", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
74772025-04-30T06:43:59.281ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
74782025-04-30T06:43:59.284ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "76160234-62d8-4135-ad1a-a7a7ced3e1b8", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
74792025-04-30T06:43:59.284ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
74802025-04-30T06:43:59.287ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9813c450-cce7-4efd-a8a4-2e699b092e21", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
74812025-04-30T06:43:59.287ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
74822025-04-30T06:43:59.289ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "88053de8-71e6-435d-a112-4df7913dc193", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
74832025-04-30T06:43:59.296ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
74842025-04-30T06:43:59.296ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
74852025-04-30T06:43:59.296ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
74862025-04-30T06:43:59.296ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
74872025-04-30T06:43:59.296ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
74882025-04-30T06:43:59.297ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
74892025-04-30T06:43:59.299ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4892559e-42f9-4953-8dba-87e2b969e76e", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
74902025-04-30T06:43:59.300ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
74912025-04-30T06:43:59.301ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74922025-04-30T06:43:59.309ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
74932025-04-30T06:43:59.309ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
74942025-04-30T06:43:59.309ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
74952025-04-30T06:43:59.309ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
74962025-04-30T06:43:59.332ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c34820c9-7ea4-4e96-ad9e-2b68c6172ecc", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:58 GMT"} })
74972025-04-30T06:43:59.332ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
74982025-04-30T06:43:59.332ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
74992025-04-30T06:43:59.332ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
75002025-04-30T06:43:59.332ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
75012025-04-30T06:43:59.333ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
75022025-04-30T06:43:59.337ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
75032025-04-30T06:43:59.337ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
75042025-04-30T06:43:59.338ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
75052025-04-30T06:43:59.339ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75062025-04-30T06:43:59.343ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
75072025-04-30T06:43:59.343ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
75082025-04-30T06:43:59.343ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
75092025-04-30T06:43:59.343ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
75102025-04-30T06:43:59.343ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
75112025-04-30T06:43:59.343ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
75122025-04-30T06:43:59.361ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75132025-04-30T06:43:59.365ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
75142025-04-30T06:43:59.365ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
75152025-04-30T06:43:59.365ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
75162025-04-30T06:43:59.366ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
75172025-04-30T06:43:59.449ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
75182025-04-30T06:43:59.449ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory
75192025-04-30T06:43:59.450ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:60772 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
75202025-04-30T06:43:59.452ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:60772 req_id = adcf00cb-290d-47e8-bcca-23741fa94988 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
75212025-04-30T06:43:59.453ZINFOtest_unauthorized (dropshot): request completed latency_us = 1108 local_addr = [::1]:57786 method = GET remote_addr = [::1]:60772 req_id = adcf00cb-290d-47e8-bcca-23741fa94988 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
75222025-04-30T06:43:59.453ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = http://[::1]:57786 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:57786/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "adcf00cb-290d-47e8-bcca-23741fa94988", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
75232025-04-30T06:43:59.456ZDEBGtest_unauthorized (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
75242025-04-30T06:43:59.456ZDEBGtest_unauthorized (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
75252025-04-30T06:43:59.456ZDEBGtest_unauthorized (BackgroundTasks): finished collection background_task = inventory_collection
75262025-04-30T06:43:59.458ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
75272025-04-30T06:43:59.458ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
75282025-04-30T06:43:59.458ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
75292025-04-30T06:43:59.458ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
75302025-04-30T06:43:59.458ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
75312025-04-30T06:43:59.462ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75322025-04-30T06:43:59.468ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
75332025-04-30T06:43:59.468ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
75342025-04-30T06:43:59.468ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
75352025-04-30T06:43:59.469ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
75362025-04-30T06:43:59.496ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
75372025-04-30T06:43:59.522ZWARNtest_unauthorized (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
75382025-04-30T06:43:59.522ZWARNtest_unauthorized (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
75392025-04-30T06:43:59.522ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 605.618255ms iteration = 23 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
75402025-04-30T06:43:59.522ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 24 reason = Signaled
75412025-04-30T06:43:59.522ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
75422025-04-30T06:43:59.522ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
75432025-04-30T06:43:59.522ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
75442025-04-30T06:43:59.522ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
75452025-04-30T06:43:59.523ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
75462025-04-30T06:43:59.526ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75472025-04-30T06:43:59.529ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
75482025-04-30T06:43:59.529ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
75492025-04-30T06:43:59.529ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
75502025-04-30T06:43:59.529ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
75512025-04-30T06:43:59.533ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1a8b2318-abf8-406d-a389-112ab91a0de8, 0)]
75522025-04-30T06:43:59.533ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
75532025-04-30T06:43:59.534ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: cb 66 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .f..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:56355 req_id = 69a8c5c0-0269-4939-abca-7f9cb6bd3365
75542025-04-30T06:43:59.534ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 52070,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:56355 req_id = 69a8c5c0-0269-4939-abca-7f9cb6bd3365
75552025-04-30T06:43:59.534ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
75562025-04-30T06:43:59.534ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
75572025-04-30T06:43:59.534ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:56355 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 69a8c5c0-0269-4939-abca-7f9cb6bd3365
75582025-04-30T06:43:59.535ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4304, tv_nsec: 334043603 } })
75592025-04-30T06:43:59.535ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
75602025-04-30T06:43:59.535ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: c2 92 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:53111 req_id = 65920930-5bfd-41c4-9557-44c9a07e4713
75612025-04-30T06:43:59.535ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 49810,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:53111 req_id = 65920930-5bfd-41c4-9557-44c9a07e4713
75622025-04-30T06:43:59.535ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
75632025-04-30T06:43:59.535ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:53111 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 65920930-5bfd-41c4-9557-44c9a07e4713
75642025-04-30T06:43:59.581ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
75652025-04-30T06:43:59.582ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
75662025-04-30T06:43:59.582ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
75672025-04-30T06:43:59.582ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
75682025-04-30T06:43:59.582ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
75692025-04-30T06:43:59.583ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
75702025-04-30T06:43:59.583ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: c5 ea 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:49881 req_id = 6802a40d-42d0-4bb1-9a4e-e64686657b9e
75712025-04-30T06:43:59.583ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 50666,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:49881 req_id = 6802a40d-42d0-4bb1-9a4e-e64686657b9e
75722025-04-30T06:43:59.584ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
75732025-04-30T06:43:59.584ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 50666, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:49881 req_id = 6802a40d-42d0-4bb1-9a4e-e64686657b9e
75742025-04-30T06:43:59.584ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
75752025-04-30T06:43:59.584ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
75762025-04-30T06:43:59.584ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
75772025-04-30T06:43:59.590ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75782025-04-30T06:43:59.609ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
75792025-04-30T06:43:59.609ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
75802025-04-30T06:43:59.609ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
75812025-04-30T06:43:59.615ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
75822025-04-30T06:43:59.626ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
75832025-04-30T06:43:59.626ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
75842025-04-30T06:43:59.627ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
75852025-04-30T06:43:59.632ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "a84dc876-7286-4af5-b83d-7faecc33faad", "content-length": "835", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
75862025-04-30T06:43:59.632ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
75872025-04-30T06:43:59.649ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "40e85fa0-af9d-42e1-a2ee-d1913b5737f4", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
75882025-04-30T06:43:59.650ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
75892025-04-30T06:43:59.663ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a1949575-c79d-41c8-a5b5-98cd1fb875df", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
75902025-04-30T06:43:59.663ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
75912025-04-30T06:43:59.668ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b13374e8-61d4-4913-84f0-eed9cacf664a", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
75922025-04-30T06:43:59.668ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
75932025-04-30T06:43:59.674ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "db95b827-0a68-4cf0-ac30-4ea34adea007", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
75942025-04-30T06:43:59.674ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
75952025-04-30T06:43:59.678ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0e3bba13-bb23-4f81-8db1-151541a20ca6", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
75962025-04-30T06:43:59.679ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
75972025-04-30T06:43:59.685ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
75982025-04-30T06:43:59.685ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7c6f8d4b-8141-434c-b63f-bc7e0a325051", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
75992025-04-30T06:43:59.685ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
76002025-04-30T06:43:59.690ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6c0063d0-bae7-4b26-b221-bb916f879b2b", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
76012025-04-30T06:43:59.690ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
76022025-04-30T06:43:59.720ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2bd986fb-8ce2-4aeb-b19c-e082f286540e", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
76032025-04-30T06:43:59.720ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
76042025-04-30T06:43:59.739ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
76052025-04-30T06:43:59.739ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
76062025-04-30T06:43:59.739ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
76072025-04-30T06:43:59.739ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
76082025-04-30T06:43:59.742ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
76092025-04-30T06:43:59.748ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0f394347-c107-4bc7-9c8b-01d343a560be", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
76102025-04-30T06:43:59.749ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
76112025-04-30T06:43:59.750ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76122025-04-30T06:43:59.758ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
76132025-04-30T06:43:59.758ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
76142025-04-30T06:43:59.758ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
76152025-04-30T06:43:59.759ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
76162025-04-30T06:43:59.764ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5e124992-4a3f-4231-882c-698b1bf6a2af", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
76172025-04-30T06:43:59.764ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
76182025-04-30T06:43:59.766ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
76192025-04-30T06:43:59.766ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
76202025-04-30T06:43:59.767ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
76212025-04-30T06:43:59.768ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6b26280b-4a29-41e8-b99d-a259fa3b4ee9", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
76222025-04-30T06:43:59.769ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
76232025-04-30T06:43:59.771ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "62d422e6-7ee6-4b04-a72f-2828bbd642df", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
76242025-04-30T06:43:59.771ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
76252025-04-30T06:43:59.777ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e3fe2f03-c0b4-4d90-8c89-484de0744f03", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
76262025-04-30T06:43:59.777ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
76272025-04-30T06:43:59.782ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "71df0a15-07c2-423d-abc4-1503794ce078", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
76282025-04-30T06:43:59.782ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
76292025-04-30T06:43:59.786ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "dd7d170c-1386-4897-acf3-bd59fe6617b9", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
76302025-04-30T06:43:59.787ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
76312025-04-30T06:43:59.789ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "20a70115-bb49-43e6-878a-3150b7e610c2", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
76322025-04-30T06:43:59.789ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
76332025-04-30T06:43:59.793ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e465c7f9-9ae9-413d-b8e4-8d261b2b21f9", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
76342025-04-30T06:43:59.793ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
76352025-04-30T06:43:59.803ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
76362025-04-30T06:43:59.803ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bf63ebc2-e085-478e-95ac-67ece9a0a029", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
76372025-04-30T06:43:59.804ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
76382025-04-30T06:43:59.807ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c26c74c1-2aee-46c0-9c9a-ab87a955858b", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
76392025-04-30T06:43:59.807ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
76402025-04-30T06:43:59.809ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e60dc61b-6f03-4570-b00f-d1b67494d353", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
76412025-04-30T06:43:59.810ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
76422025-04-30T06:43:59.813ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1b0eb949-bb80-4c13-a5f1-043576671de9", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
76432025-04-30T06:43:59.813ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
76442025-04-30T06:43:59.814ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1ca06b39-fea8-4559-8670-44b98ce47e7b", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
76452025-04-30T06:43:59.814ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
76462025-04-30T06:43:59.816ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c6c81c77-ee32-4694-9d35-3924e0c34657", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
76472025-04-30T06:43:59.816ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
76482025-04-30T06:43:59.820ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "501f3187-c770-40c1-945c-d581a07b9912", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
76492025-04-30T06:43:59.820ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
76502025-04-30T06:43:59.825ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "176909c5-0ab0-46d3-a48a-c987c8b09df8", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
76512025-04-30T06:43:59.825ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
76522025-04-30T06:43:59.827ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e963bb07-3a8f-4ca2-b5ef-df64936eda9d", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
76532025-04-30T06:43:59.827ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
76542025-04-30T06:43:59.830ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "dc978a53-1a1e-4980-a924-ffcc380f4049", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
76552025-04-30T06:43:59.830ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
76562025-04-30T06:43:59.833ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "61e68762-92f8-40fa-b04b-3b7b0d6595a1", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
76572025-04-30T06:43:59.833ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
76582025-04-30T06:43:59.844ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5031fa22-c529-4b87-8eec-1ebe3fce28d4", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
76592025-04-30T06:43:59.844ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
76602025-04-30T06:43:59.844ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "48de55a2-1161-4bee-8d97-2902424dce8c", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
76612025-04-30T06:43:59.845ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
76622025-04-30T06:43:59.845ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fdb8938d-63b3-4e13-8f07-97e7560291a2", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
76632025-04-30T06:43:59.845ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
76642025-04-30T06:43:59.845ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1ddd28d0-fdb7-49f7-a907-e9a5cc5408f3", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
76652025-04-30T06:43:59.845ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
76662025-04-30T06:43:59.845ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3afed8d1-21e2-4748-8efa-f7468df10995", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
76672025-04-30T06:43:59.845ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
76682025-04-30T06:43:59.845ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "04fe5478-8fb8-411f-b141-b336acd6cd1e", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
76692025-04-30T06:43:59.846ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
76702025-04-30T06:43:59.849ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c97f75a2-bde0-4796-b21b-a05f759952dd", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
76712025-04-30T06:43:59.849ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
76722025-04-30T06:43:59.851ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "df3e0dd7-3c4e-45a1-a5fd-06914acc5bd5", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
76732025-04-30T06:43:59.851ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
76742025-04-30T06:43:59.853ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "68bc2ef4-75b6-471e-a49b-a42cecb49db6", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
76752025-04-30T06:43:59.853ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
76762025-04-30T06:43:59.854ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
76772025-04-30T06:43:59.854ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
76782025-04-30T06:43:59.854ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
76792025-04-30T06:43:59.854ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
76802025-04-30T06:43:59.855ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
76812025-04-30T06:43:59.861ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76822025-04-30T06:43:59.872ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
76832025-04-30T06:43:59.872ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
76842025-04-30T06:43:59.872ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
76852025-04-30T06:43:59.873ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
76862025-04-30T06:43:59.875ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "61f5a444-031a-4c0a-8b0b-3c33ede3636a", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
76872025-04-30T06:43:59.875ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
76882025-04-30T06:43:59.876ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
76892025-04-30T06:43:59.876ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
76902025-04-30T06:43:59.876ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
76912025-04-30T06:43:59.881ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f36ed5fc-89df-4e75-9882-ccca5402a457", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
76922025-04-30T06:43:59.882ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
76932025-04-30T06:43:59.887ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "dbaad53d-c1a8-4fef-9c6d-570522eab829", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
76942025-04-30T06:43:59.887ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
76952025-04-30T06:43:59.895ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "11a416f9-c240-4da5-a725-54e0454d17f8", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
76962025-04-30T06:43:59.895ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
76972025-04-30T06:43:59.895ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1b91afa4-a37d-4aae-996f-dfeeb4e87994", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
76982025-04-30T06:43:59.896ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
76992025-04-30T06:43:59.897ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "700dd8c6-c3ba-4978-89e5-6a480394ec88", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
77002025-04-30T06:43:59.897ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
77012025-04-30T06:43:59.898ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5b48008c-1292-416b-9fc7-9e68a5c46e18", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
77022025-04-30T06:43:59.898ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
77032025-04-30T06:43:59.898ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
77042025-04-30T06:43:59.898ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
77052025-04-30T06:43:59.898ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
77062025-04-30T06:43:59.899ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
77072025-04-30T06:43:59.903ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77082025-04-30T06:43:59.908ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
77092025-04-30T06:43:59.908ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
77102025-04-30T06:43:59.908ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
77112025-04-30T06:43:59.908ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
77122025-04-30T06:43:59.908ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
77132025-04-30T06:43:59.915ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
77142025-04-30T06:43:59.918ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77152025-04-30T06:43:59.929ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
77162025-04-30T06:43:59.945ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
77172025-04-30T06:43:59.945ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
77182025-04-30T06:43:59.946ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
77192025-04-30T06:44:00.071ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
77202025-04-30T06:44:00.072ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory
77212025-04-30T06:44:00.072ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
77222025-04-30T06:44:00.073ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:61859 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
77232025-04-30T06:44:00.074ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:61859 req_id = 0ae4f62a-b041-41c2-8f38-06bd2308282e sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
77242025-04-30T06:44:00.074ZINFOtest_unauthorized (dropshot): request completed latency_us = 480 local_addr = [::1]:57786 method = GET remote_addr = [::1]:61859 req_id = 0ae4f62a-b041-41c2-8f38-06bd2308282e response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
77252025-04-30T06:44:00.075ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = http://[::1]:57786 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:57786/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "0ae4f62a-b041-41c2-8f38-06bd2308282e", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
77262025-04-30T06:44:00.077ZDEBGtest_unauthorized (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
77272025-04-30T06:44:00.077ZDEBGtest_unauthorized (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
77282025-04-30T06:44:00.077ZDEBGtest_unauthorized (BackgroundTasks): finished collection background_task = inventory_collection
77292025-04-30T06:44:00.081ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
77302025-04-30T06:44:00.081ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
77312025-04-30T06:44:00.081ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
77322025-04-30T06:44:00.081ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
77332025-04-30T06:44:00.082ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
77342025-04-30T06:44:00.086ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77352025-04-30T06:44:00.092ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
77362025-04-30T06:44:00.093ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
77372025-04-30T06:44:00.093ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
77382025-04-30T06:44:00.093ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
77392025-04-30T06:44:00.125ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
77402025-04-30T06:44:00.125ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
77412025-04-30T06:44:00.125ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
77422025-04-30T06:44:00.125ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
77432025-04-30T06:44:00.128ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
77442025-04-30T06:44:00.130ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77452025-04-30T06:44:00.136ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
77462025-04-30T06:44:00.136ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
77472025-04-30T06:44:00.136ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
77482025-04-30T06:44:00.137ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
77492025-04-30T06:44:00.140ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
77502025-04-30T06:44:00.140ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
77512025-04-30T06:44:00.141ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
77522025-04-30T06:44:00.141ZWARNtest_unauthorized (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
77532025-04-30T06:44:00.142ZWARNtest_unauthorized (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
77542025-04-30T06:44:00.142ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 619.552317ms iteration = 24 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
77552025-04-30T06:44:00.142ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 25 reason = Signaled
77562025-04-30T06:44:00.142ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
77572025-04-30T06:44:00.142ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
77582025-04-30T06:44:00.142ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
77592025-04-30T06:44:00.142ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
77602025-04-30T06:44:00.142ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
77612025-04-30T06:44:00.145ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77622025-04-30T06:44:00.153ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
77632025-04-30T06:44:00.153ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
77642025-04-30T06:44:00.153ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
77652025-04-30T06:44:00.154ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
77662025-04-30T06:44:00.162ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1a8b2318-abf8-406d-a389-112ab91a0de8, 0)]
77672025-04-30T06:44:00.162ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
77682025-04-30T06:44:00.164ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 00 ce 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:51041 req_id = c2034e5c-639d-4002-97b7-4acf4c70720c
77692025-04-30T06:44:00.164ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 206,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51041 req_id = c2034e5c-639d-4002-97b7-4acf4c70720c
77702025-04-30T06:44:00.164ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
77712025-04-30T06:44:00.164ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
77722025-04-30T06:44:00.164ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:51041 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = c2034e5c-639d-4002-97b7-4acf4c70720c
77732025-04-30T06:44:00.165ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4304, tv_nsec: 964229126 } })
77742025-04-30T06:44:00.165ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
77752025-04-30T06:44:00.165ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 07 a8 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:49743 req_id = a9d5ea2b-6719-482f-9a70-7651ab859f2b
77762025-04-30T06:44:00.165ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 1960,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:49743 req_id = a9d5ea2b-6719-482f-9a70-7651ab859f2b
77772025-04-30T06:44:00.165ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
77782025-04-30T06:44:00.165ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:49743 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = a9d5ea2b-6719-482f-9a70-7651ab859f2b
77792025-04-30T06:44:00.221ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
77802025-04-30T06:44:00.222ZDEBGtest_unauthorized (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
77812025-04-30T06:44:00.223ZTRACtest_unauthorized (results-sink): batch interval expired, but no samples to insert collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
77822025-04-30T06:44:00.223ZDEBGtest_unauthorized (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
77832025-04-30T06:44:00.223ZDEBGtest_unauthorized (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
77842025-04-30T06:44:00.225ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 33 bc 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c 3............_cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:65157 req_id = 220e3f8e-19c4-4300-8ee7-41c8fae35a15
77852025-04-30T06:44:00.225ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 13244,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:65157 req_id = 220e3f8e-19c4-4300-8ee7-41c8fae35a15
77862025-04-30T06:44:00.225ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
77872025-04-30T06:44:00.225ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 13244, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:65157 req_id = 220e3f8e-19c4-4300-8ee7-41c8fae35a15
77882025-04-30T06:44:00.227ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
77892025-04-30T06:44:00.227ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
77902025-04-30T06:44:00.227ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
77912025-04-30T06:44:00.228ZTRACtest_unauthorized (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
77922025-04-30T06:44:00.229ZDEBGtest_unauthorized (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
77932025-04-30T06:44:00.229ZTRACtest_unauthorized (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
77942025-04-30T06:44:00.229ZTRACtest_unauthorized (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
77952025-04-30T06:44:00.235ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "0791487e-b15b-4b47-9e9a-f1e4640bd7c1", "content-length": "835", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
77962025-04-30T06:44:00.235ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
77972025-04-30T06:44:00.238ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a676f9f0-b362-47dc-8af0-6c47358c47a6", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
77982025-04-30T06:44:00.238ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
77992025-04-30T06:44:00.242ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bf73fc5c-7e8b-4af5-8997-e6a27bbf3edc", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
78002025-04-30T06:44:00.243ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
78012025-04-30T06:44:00.248ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
78022025-04-30T06:44:00.252ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fed6ba90-c46d-4715-a4a8-0fbbab20eb5f", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
78032025-04-30T06:44:00.252ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
78042025-04-30T06:44:00.255ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "470df1ba-4550-4e0f-aaa3-c10730af6a94", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
78052025-04-30T06:44:00.255ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
78062025-04-30T06:44:00.259ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "af30b066-bbbf-4f05-af06-35edcbf5c3b1", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
78072025-04-30T06:44:00.259ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
78082025-04-30T06:44:00.265ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "02ad4717-e803-425e-8656-0b852623dac3", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
78092025-04-30T06:44:00.265ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
78102025-04-30T06:44:00.269ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "33e2d423-7219-4c60-a074-fa6694fc52a7", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
78112025-04-30T06:44:00.269ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
78122025-04-30T06:44:00.270ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4f10a51b-a827-4b6b-809f-168ebe75ce01", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
78132025-04-30T06:44:00.271ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
78142025-04-30T06:44:00.272ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c93e4bab-8ec9-49a7-a810-c684752883ed", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
78152025-04-30T06:44:00.272ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
78162025-04-30T06:44:00.274ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "323bb76b-2414-422e-b1e9-b2b83dcb4d7a", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
78172025-04-30T06:44:00.274ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
78182025-04-30T06:44:00.283ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a261b5c5-67cb-4ea2-8021-e817c37afa46", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
78192025-04-30T06:44:00.284ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
78202025-04-30T06:44:00.286ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7cec8d65-caf0-42e3-ad4d-60de172f5abf", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
78212025-04-30T06:44:00.288ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
78222025-04-30T06:44:00.290ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f4f0e085-0167-47e8-9d4d-b558a684e4bf", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
78232025-04-30T06:44:00.290ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
78242025-04-30T06:44:00.292ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "153c011e-a89d-48b6-8a86-9cd781f839e9", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
78252025-04-30T06:44:00.292ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
78262025-04-30T06:44:00.294ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b55b3d18-3353-4874-b87b-80d28d4b5138", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
78272025-04-30T06:44:00.294ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
78282025-04-30T06:44:00.299ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e0a7493c-c732-4f2b-8359-9fa2c3d7b8ec", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
78292025-04-30T06:44:00.300ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
78302025-04-30T06:44:00.300ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
78312025-04-30T06:44:00.300ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
78322025-04-30T06:44:00.300ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
78332025-04-30T06:44:00.300ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
78342025-04-30T06:44:00.301ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
78352025-04-30T06:44:00.303ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d71ca66e-da96-428e-9804-947f63a4f60f", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
78362025-04-30T06:44:00.303ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
78372025-04-30T06:44:00.303ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78382025-04-30T06:44:00.311ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
78392025-04-30T06:44:00.311ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
78402025-04-30T06:44:00.311ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
78412025-04-30T06:44:00.311ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
78422025-04-30T06:44:00.314ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
78432025-04-30T06:44:00.314ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
78442025-04-30T06:44:00.315ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
78452025-04-30T06:44:00.315ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8f8ad3c4-d83b-4b2e-9e68-7d47b3e6c45a", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
78462025-04-30T06:44:00.315ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
78472025-04-30T06:44:00.318ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1c17c60b-4cf8-4949-bcf4-76c44d10b3da", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
78482025-04-30T06:44:00.318ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
78492025-04-30T06:44:00.319ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1c4ae5a8-238a-4631-9178-18dcf1ee6b87", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
78502025-04-30T06:44:00.320ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
78512025-04-30T06:44:00.321ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "277e7e99-5ed4-4ac7-be33-662ee56dac49", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
78522025-04-30T06:44:00.321ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
78532025-04-30T06:44:00.322ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c774a14a-1f3c-4703-abbf-9c6d6d41a7ac", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
78542025-04-30T06:44:00.323ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
78552025-04-30T06:44:00.324ZWARNtest_unauthorized (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
78562025-04-30T06:44:00.325ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9bb1642e-0294-4a9d-9549-ce4b4715c71e", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
78572025-04-30T06:44:00.325ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
78582025-04-30T06:44:00.329ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7c2f50a7-1b17-44b0-abd0-0d1cfc2057a2", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
78592025-04-30T06:44:00.329ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
78602025-04-30T06:44:00.337ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "530bc168-20b1-46db-9932-d36a6337633c", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
78612025-04-30T06:44:00.338ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
78622025-04-30T06:44:00.342ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1db10a6a-23a7-43f2-a303-d4569ac05460", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
78632025-04-30T06:44:00.342ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
78642025-04-30T06:44:00.352ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e0cf77ba-1f9d-4548-b346-b7c0c7fdd9c9", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
78652025-04-30T06:44:00.352ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
78662025-04-30T06:44:00.353ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
78672025-04-30T06:44:00.358ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a18c15ba-6e93-4c91-bad2-1dba20703c2c", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
78682025-04-30T06:44:00.358ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
78692025-04-30T06:44:00.366ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "477daab2-ab2c-4b9a-92d8-98abb2eae987", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
78702025-04-30T06:44:00.366ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
78712025-04-30T06:44:00.367ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6ac13d2d-d5a2-4c6d-bf5c-0e5b2d66ca74", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
78722025-04-30T06:44:00.368ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
78732025-04-30T06:44:00.369ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5720740b-c6fb-4086-b57f-0fe9c35ab7e3", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
78742025-04-30T06:44:00.369ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
78752025-04-30T06:44:00.374ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2ba98ab9-c9fd-467a-bd1a-8b9c048afade", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
78762025-04-30T06:44:00.374ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
78772025-04-30T06:44:00.377ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2ad9ba30-4b6d-40fb-b617-ec86a4e47c77", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
78782025-04-30T06:44:00.377ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
78792025-04-30T06:44:00.381ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2a898cf1-8708-436f-88fe-90df90a1e1ba", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
78802025-04-30T06:44:00.381ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
78812025-04-30T06:44:00.386ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d38f5750-1f7b-4356-8f2e-3eb520799d86", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
78822025-04-30T06:44:00.386ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
78832025-04-30T06:44:00.390ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "65fc1aaa-e2a8-4344-b309-b8dd9a6e6b85", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
78842025-04-30T06:44:00.390ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
78852025-04-30T06:44:00.392ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a7071a6-8fa9-4ab1-b325-0272a1f78c10", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
78862025-04-30T06:44:00.392ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
78872025-04-30T06:44:00.394ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "85e574d6-94a7-4702-a744-a44bf3d73cba", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
78882025-04-30T06:44:00.394ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
78892025-04-30T06:44:00.397ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "79ee01dd-8fca-4bdf-a4b6-f4bd09a499f1", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
78902025-04-30T06:44:00.397ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
78912025-04-30T06:44:00.402ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b56b6343-f8ed-455b-bebe-ecca4386f74c", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
78922025-04-30T06:44:00.402ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
78932025-04-30T06:44:00.403ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b5194b6e-ecb8-45f4-aab5-66555901046c", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
78942025-04-30T06:44:00.404ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
78952025-04-30T06:44:00.405ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
78962025-04-30T06:44:00.405ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
78972025-04-30T06:44:00.406ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
78982025-04-30T06:44:00.406ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
78992025-04-30T06:44:00.407ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
79002025-04-30T06:44:00.410ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d4b6237f-bba2-4ec8-a7fa-816e1eec6975", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
79012025-04-30T06:44:00.410ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
79022025-04-30T06:44:00.411ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79032025-04-30T06:44:00.417ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
79042025-04-30T06:44:00.417ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
79052025-04-30T06:44:00.417ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
79062025-04-30T06:44:00.418ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
79072025-04-30T06:44:00.419ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9dc89044-a908-4c77-abc7-b977b1964f7c", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
79082025-04-30T06:44:00.420ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
79092025-04-30T06:44:00.421ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
79102025-04-30T06:44:00.421ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
79112025-04-30T06:44:00.421ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
79122025-04-30T06:44:00.422ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2357e804-ae82-4c61-a6f0-384235fd1187", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
79132025-04-30T06:44:00.422ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
79142025-04-30T06:44:00.422ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
79152025-04-30T06:44:00.422ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
79162025-04-30T06:44:00.422ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
79172025-04-30T06:44:00.423ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
79182025-04-30T06:44:00.425ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79192025-04-30T06:44:00.430ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
79202025-04-30T06:44:00.430ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
79212025-04-30T06:44:00.430ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
79222025-04-30T06:44:00.430ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
79232025-04-30T06:44:00.430ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
79242025-04-30T06:44:00.431ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
79252025-04-30T06:44:00.434ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79262025-04-30T06:44:00.440ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
79272025-04-30T06:44:00.440ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
79282025-04-30T06:44:00.440ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
79292025-04-30T06:44:00.440ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
79302025-04-30T06:44:00.501ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
79312025-04-30T06:44:00.501ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory
79322025-04-30T06:44:00.502ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:62789 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
79332025-04-30T06:44:00.503ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:62789 req_id = bb26dee1-11dd-44fe-9cac-9587f0886344 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
79342025-04-30T06:44:00.509ZINFOtest_unauthorized (dropshot): request completed latency_us = 6808 local_addr = [::1]:57786 method = GET remote_addr = [::1]:62789 req_id = bb26dee1-11dd-44fe-9cac-9587f0886344 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
79352025-04-30T06:44:00.518ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = http://[::1]:57786 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:57786/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb26dee1-11dd-44fe-9cac-9587f0886344", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
79362025-04-30T06:44:00.522ZDEBGtest_unauthorized (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
79372025-04-30T06:44:00.522ZDEBGtest_unauthorized (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
79382025-04-30T06:44:00.522ZDEBGtest_unauthorized (BackgroundTasks): finished collection background_task = inventory_collection
79392025-04-30T06:44:00.525ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
79402025-04-30T06:44:00.525ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
79412025-04-30T06:44:00.525ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
79422025-04-30T06:44:00.525ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
79432025-04-30T06:44:00.526ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
79442025-04-30T06:44:00.540ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79452025-04-30T06:44:00.545ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
79462025-04-30T06:44:00.547ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
79472025-04-30T06:44:00.547ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
79482025-04-30T06:44:00.548ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
79492025-04-30T06:44:00.566ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
79502025-04-30T06:44:00.620ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
79512025-04-30T06:44:00.620ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
79522025-04-30T06:44:00.620ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
79532025-04-30T06:44:00.620ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
79542025-04-30T06:44:00.621ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
79552025-04-30T06:44:00.625ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79562025-04-30T06:44:00.631ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
79572025-04-30T06:44:00.631ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
79582025-04-30T06:44:00.631ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
79592025-04-30T06:44:00.631ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
79602025-04-30T06:44:00.633ZWARNtest_unauthorized (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
79612025-04-30T06:44:00.633ZWARNtest_unauthorized (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
79622025-04-30T06:44:00.633ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 491.545662ms iteration = 25 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
79632025-04-30T06:44:00.633ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 26 reason = Signaled
79642025-04-30T06:44:00.633ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
79652025-04-30T06:44:00.633ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
79662025-04-30T06:44:00.634ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
79672025-04-30T06:44:00.634ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
79682025-04-30T06:44:00.635ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
79692025-04-30T06:44:00.635ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
79702025-04-30T06:44:00.635ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
79712025-04-30T06:44:00.635ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
79722025-04-30T06:44:00.638ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79732025-04-30T06:44:00.645ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
79742025-04-30T06:44:00.645ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
79752025-04-30T06:44:00.645ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
79762025-04-30T06:44:00.651ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
79772025-04-30T06:44:00.662ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1a8b2318-abf8-406d-a389-112ab91a0de8, 0)]
79782025-04-30T06:44:00.662ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
79792025-04-30T06:44:00.662ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 8f 26 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .&..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:52235 req_id = b1f77fa5-949a-4252-ab56-04873c679062
79802025-04-30T06:44:00.662ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 36646,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52235 req_id = b1f77fa5-949a-4252-ab56-04873c679062
79812025-04-30T06:44:00.663ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
79822025-04-30T06:44:00.663ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
79832025-04-30T06:44:00.663ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:52235 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = b1f77fa5-949a-4252-ab56-04873c679062
79842025-04-30T06:44:00.665ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4305, tv_nsec: 464756154 } })
79852025-04-30T06:44:00.665ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
79862025-04-30T06:44:00.668ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: f6 a0 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:58895 req_id = 95b5e66f-82b6-492f-a56b-1dc83e1128f8
79872025-04-30T06:44:00.668ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 63136,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:58895 req_id = 95b5e66f-82b6-492f-a56b-1dc83e1128f8
79882025-04-30T06:44:00.668ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
79892025-04-30T06:44:00.668ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:58895 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 95b5e66f-82b6-492f-a56b-1dc83e1128f8
79902025-04-30T06:44:00.759ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
79912025-04-30T06:44:00.761ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 2e 04 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:50144 req_id = 57681957-bc82-4992-9521-3f3d6366b3df
79922025-04-30T06:44:00.784ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 11780,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:50144 req_id = 57681957-bc82-4992-9521-3f3d6366b3df
79932025-04-30T06:44:00.784ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
79942025-04-30T06:44:00.785ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 11780, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:50144 req_id = 57681957-bc82-4992-9521-3f3d6366b3df
79952025-04-30T06:44:00.786ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
79962025-04-30T06:44:00.786ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
79972025-04-30T06:44:00.786ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
79982025-04-30T06:44:00.834ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "d48f0df9-774f-427b-934c-32ca70e50795", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
79992025-04-30T06:44:00.835ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
80002025-04-30T06:44:00.839ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ef946fa0-1fd2-4a03-b317-553513b4b7d4", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
80012025-04-30T06:44:00.839ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
80022025-04-30T06:44:00.843ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f9f1fe5b-3bb8-457b-a26f-1bdf7d856d60", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
80032025-04-30T06:44:00.843ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
80042025-04-30T06:44:00.846ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
80052025-04-30T06:44:00.848ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bddf4d39-f84c-4733-a432-a7de50eba150", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
80062025-04-30T06:44:00.849ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
80072025-04-30T06:44:00.853ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0344ca8b-1197-4397-af83-e5cb65626ec4", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
80082025-04-30T06:44:00.854ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
80092025-04-30T06:44:00.856ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c103ad0b-5aba-41d5-85f3-f47e76b9485c", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
80102025-04-30T06:44:00.856ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
80112025-04-30T06:44:00.878ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2d804458-d9b2-41b0-b454-6e0d269843e5", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
80122025-04-30T06:44:00.879ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
80132025-04-30T06:44:00.887ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f3f5fb33-ec1a-467c-9dfd-8baebcc87a19", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
80142025-04-30T06:44:00.887ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
80152025-04-30T06:44:00.889ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b9219e7e-5b29-490f-ad7e-747af1ca33f2", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
80162025-04-30T06:44:00.889ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
80172025-04-30T06:44:00.893ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "660fecd8-e101-4025-87b1-4587dc0b4009", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
80182025-04-30T06:44:00.894ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
80192025-04-30T06:44:00.903ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
80202025-04-30T06:44:00.904ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
80212025-04-30T06:44:00.905ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
80222025-04-30T06:44:00.905ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
80232025-04-30T06:44:00.906ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
80242025-04-30T06:44:00.907ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e2a2f79b-6667-4d06-b0ce-64bbf4a04a27", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
80252025-04-30T06:44:00.908ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
80262025-04-30T06:44:00.910ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80272025-04-30T06:44:00.963ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
80282025-04-30T06:44:00.975ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
80292025-04-30T06:44:00.976ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
80302025-04-30T06:44:00.976ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
80312025-04-30T06:44:00.980ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ebf85a57-8040-48cf-bcca-6147df3c7625", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
80322025-04-30T06:44:00.980ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
80332025-04-30T06:44:00.980ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
80342025-04-30T06:44:00.980ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
80352025-04-30T06:44:00.981ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
80362025-04-30T06:44:00.984ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "87b9cf30-4d03-4cd0-9213-da0015f5aa44", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
80372025-04-30T06:44:00.984ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
80382025-04-30T06:44:00.988ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fd5a0ab6-934f-4fd7-aa98-5ba64ddf82d2", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
80392025-04-30T06:44:00.988ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
80402025-04-30T06:44:00.992ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ebaa37e7-87b6-4cbe-9c96-013d2be72c5b", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
80412025-04-30T06:44:00.992ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
80422025-04-30T06:44:00.997ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d7264985-ca62-42a4-a233-7ef57ea6c48b", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
80432025-04-30T06:44:00.997ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
80442025-04-30T06:44:01.006ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "651904d6-2198-4e41-b694-72417e4aae8f", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
80452025-04-30T06:44:01.006ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
80462025-04-30T06:44:01.010ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
80472025-04-30T06:44:01.012ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4b424fd9-a581-4ae7-a368-2e797e6ad5e7", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
80482025-04-30T06:44:01.012ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
80492025-04-30T06:44:01.016ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d01f286e-a999-4efc-9e5e-8f0804e259aa", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
80502025-04-30T06:44:01.016ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
80512025-04-30T06:44:01.017ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e8394cc7-3178-4d29-8c97-f6e23db7ecaf", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
80522025-04-30T06:44:01.017ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
80532025-04-30T06:44:01.019ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2bf31fc9-54be-4cab-8bf3-93ca53510ccf", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
80542025-04-30T06:44:01.019ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
80552025-04-30T06:44:01.020ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a0252549-68a7-4c6b-a291-8c7e7c301671", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
80562025-04-30T06:44:01.022ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
80572025-04-30T06:44:01.023ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "541bf7c0-7dbd-49f0-8be5-dc2d246d64d0", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
80582025-04-30T06:44:01.023ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
80592025-04-30T06:44:01.025ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a1d70015-f65f-4bcb-b689-60cde9379134", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
80602025-04-30T06:44:01.025ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
80612025-04-30T06:44:01.029ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "41eb1a0e-98f3-41e2-8b48-08c9bf1c2b8d", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
80622025-04-30T06:44:01.029ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
80632025-04-30T06:44:01.031ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1ce57982-d405-48f7-97be-52633ac2a6dc", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
80642025-04-30T06:44:01.032ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
80652025-04-30T06:44:01.034ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa25048f-b6fe-40ec-ab4c-88528edb3c14", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
80662025-04-30T06:44:01.034ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
80672025-04-30T06:44:01.036ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2aeb6e48-36d7-4ecc-a6bf-8f798b2902eb", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
80682025-04-30T06:44:01.036ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
80692025-04-30T06:44:01.039ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1e9dc48c-1d1f-45a9-afdf-d1a62b8d8b49", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
80702025-04-30T06:44:01.041ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
80712025-04-30T06:44:01.044ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6220b38c-8092-48f7-96dd-68c18ee5b94e", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
80722025-04-30T06:44:01.044ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
80732025-04-30T06:44:01.046ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5b0ef2d1-b6ee-4474-9973-1c9fe7dc1e82", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
80742025-04-30T06:44:01.046ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
80752025-04-30T06:44:01.048ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ed9eb903-debf-4497-ab33-0ffab8b2e2ff", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
80762025-04-30T06:44:01.049ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
80772025-04-30T06:44:01.050ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "bbdc915b-3548-4332-b197-febe577a49b8", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
80782025-04-30T06:44:01.050ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
80792025-04-30T06:44:01.053ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ac840eaa-5ddb-4f7a-87b4-e8054e65cf42", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
80802025-04-30T06:44:01.054ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
80812025-04-30T06:44:01.055ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "687aefdb-8d6c-475e-af44-5e3ea5d7b72a", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
80822025-04-30T06:44:01.055ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
80832025-04-30T06:44:01.058ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "85e0fc5b-ccef-46ca-bd3a-db7f598e2c54", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
80842025-04-30T06:44:01.058ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
80852025-04-30T06:44:01.060ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "35be90ca-e7d9-4025-a07a-b176561db11d", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
80862025-04-30T06:44:01.060ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
80872025-04-30T06:44:01.061ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
80882025-04-30T06:44:01.061ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
80892025-04-30T06:44:01.061ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
80902025-04-30T06:44:01.061ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
80912025-04-30T06:44:01.062ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
80922025-04-30T06:44:01.064ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ead0ea10-d7ea-4232-81cc-81fa1ae4e940", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
80932025-04-30T06:44:01.064ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
80942025-04-30T06:44:01.064ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80952025-04-30T06:44:01.076ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
80962025-04-30T06:44:01.076ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
80972025-04-30T06:44:01.076ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
80982025-04-30T06:44:01.077ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
80992025-04-30T06:44:01.096ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
81002025-04-30T06:44:01.097ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
81012025-04-30T06:44:01.099ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
81022025-04-30T06:44:01.100ZINFOtest_unauthorized (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
81032025-04-30T06:44:01.102ZDEBGtest_unauthorized (oximeter): client request body = None method = GET uri = http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
81042025-04-30T06:44:01.103ZTRACtest_unauthorized (dropshot_internal): incoming request local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 203efd0c-6a97-496a-ae7c-eb6e5c674a6f uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
81052025-04-30T06:44:01.104ZTRACtest_unauthorized (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 203efd0c-6a97-496a-ae7c-eb6e5c674a6f resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
81062025-04-30T06:44:01.105ZDEBGtest_unauthorized (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 203efd0c-6a97-496a-ae7c-eb6e5c674a6f roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
81072025-04-30T06:44:01.106ZDEBGtest_unauthorized (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 203efd0c-6a97-496a-ae7c-eb6e5c674a6f resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
81082025-04-30T06:44:01.107ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6c9a12af-c16c-44f2-afce-2961294fe99a", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
81092025-04-30T06:44:01.108ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
81102025-04-30T06:44:01.110ZINFOtest_unauthorized (dropshot_internal): request completed latency_us = 6455 local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 203efd0c-6a97-496a-ae7c-eb6e5c674a6f response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
81112025-04-30T06:44:01.113ZDEBGtest_unauthorized (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "203efd0c-6a97-496a-ae7c-eb6e5c674a6f", "content-length": "405", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
81122025-04-30T06:44:01.113ZDEBGtest_unauthorized (oximeter): client request body = None method = GET uri = http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
81132025-04-30T06:44:01.114ZTRACtest_unauthorized (dropshot_internal): incoming request local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = aa9a009a-cb46-439d-9bd3-857d14cc7f12 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
81142025-04-30T06:44:01.114ZTRACtest_unauthorized (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = aa9a009a-cb46-439d-9bd3-857d14cc7f12 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
81152025-04-30T06:44:01.115ZDEBGtest_unauthorized (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = aa9a009a-cb46-439d-9bd3-857d14cc7f12 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
81162025-04-30T06:44:01.117ZDEBGtest_unauthorized (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = aa9a009a-cb46-439d-9bd3-857d14cc7f12 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
81172025-04-30T06:44:01.117ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "96adca50-fa3c-42fd-94bc-4fbf8e8e437c", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
81182025-04-30T06:44:01.118ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
81192025-04-30T06:44:01.120ZINFOtest_unauthorized (dropshot_internal): request completed latency_us = 6222 local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = aa9a009a-cb46-439d-9bd3-857d14cc7f12 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
81202025-04-30T06:44:01.121ZDEBGtest_unauthorized (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "aa9a009a-cb46-439d-9bd3-857d14cc7f12", "content-length": "29", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
81212025-04-30T06:44:01.121ZTRACtest_unauthorized (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 8d752adb-08b9-405f-a11b-66019bf447b5
81222025-04-30T06:44:01.121ZTRACtest_unauthorized (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
81232025-04-30T06:44:01.121ZINFOtest_unauthorized (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
81242025-04-30T06:44:01.133ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6639a946-cbe4-45d8-b080-d8d39fbd360e", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
81252025-04-30T06:44:01.133ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
81262025-04-30T06:44:01.147ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "31e9a8b9-903c-43a6-acd6-c688b610fec6", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
81272025-04-30T06:44:01.147ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
81282025-04-30T06:44:01.149ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "396b5be0-b44f-4ec3-aba0-96343fa0e16a", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
81292025-04-30T06:44:01.149ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
81302025-04-30T06:44:01.151ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "db74bf88-c58d-4e2c-bacd-a80dd5ecdf53", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
81312025-04-30T06:44:01.151ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
81322025-04-30T06:44:01.152ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cce0f920-0327-4537-993d-80c786ff0f9d", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
81332025-04-30T06:44:01.152ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
81342025-04-30T06:44:01.153ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
81352025-04-30T06:44:01.153ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
81362025-04-30T06:44:01.153ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
81372025-04-30T06:44:01.154ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
81382025-04-30T06:44:01.172ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81392025-04-30T06:44:01.178ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
81402025-04-30T06:44:01.178ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
81412025-04-30T06:44:01.178ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
81422025-04-30T06:44:01.178ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
81432025-04-30T06:44:01.178ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
81442025-04-30T06:44:01.179ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
81452025-04-30T06:44:01.180ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
81462025-04-30T06:44:01.180ZDEBGtest_unauthorized (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
81472025-04-30T06:44:01.180ZDEBGtest_unauthorized (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
81482025-04-30T06:44:01.181ZDEBGtest_unauthorized (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a
81492025-04-30T06:44:01.182ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
81502025-04-30T06:44:01.183ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81512025-04-30T06:44:01.189ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
81522025-04-30T06:44:01.190ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
81532025-04-30T06:44:01.190ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
81542025-04-30T06:44:01.190ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
81552025-04-30T06:44:01.194ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 2 table_name = fields_string
81562025-04-30T06:44:01.195ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
81572025-04-30T06:44:01.275ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
81582025-04-30T06:44:01.275ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory
81592025-04-30T06:44:01.275ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
81602025-04-30T06:44:01.275ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
81612025-04-30T06:44:01.275ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
81622025-04-30T06:44:01.275ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
81632025-04-30T06:44:01.276ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
81642025-04-30T06:44:01.276ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:58632 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
81652025-04-30T06:44:01.277ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 1 table_name = measurements_i64
81662025-04-30T06:44:01.277ZTRACtest_unauthorized (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
81672025-04-30T06:44:01.277ZDEBGtest_unauthorized (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
81682025-04-30T06:44:01.277ZDEBGtest_unauthorized (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
81692025-04-30T06:44:01.280ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:58632 req_id = 58713c6d-3520-4a46-9d4a-fa5cbf347c21 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
81702025-04-30T06:44:01.282ZINFOtest_unauthorized (dropshot): request completed latency_us = 1428 local_addr = [::1]:57786 method = GET remote_addr = [::1]:58632 req_id = 58713c6d-3520-4a46-9d4a-fa5cbf347c21 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
81712025-04-30T06:44:01.282ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81722025-04-30T06:44:01.286ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
81732025-04-30T06:44:01.286ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
81742025-04-30T06:44:01.286ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
81752025-04-30T06:44:01.287ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
81762025-04-30T06:44:01.287ZTRACtest_unauthorized (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
81772025-04-30T06:44:01.288ZDEBGtest_unauthorized (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
81782025-04-30T06:44:01.288ZTRACtest_unauthorized (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
81792025-04-30T06:44:01.288ZTRACtest_unauthorized (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
81802025-04-30T06:44:01.288ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = http://[::1]:57786 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:57786/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "58713c6d-3520-4a46-9d4a-fa5cbf347c21", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
81812025-04-30T06:44:01.290ZDEBGtest_unauthorized (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
81822025-04-30T06:44:01.290ZDEBGtest_unauthorized (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
81832025-04-30T06:44:01.290ZDEBGtest_unauthorized (BackgroundTasks): finished collection background_task = inventory_collection
81842025-04-30T06:44:01.292ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
81852025-04-30T06:44:01.292ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
81862025-04-30T06:44:01.293ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
81872025-04-30T06:44:01.293ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
81882025-04-30T06:44:01.293ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
81892025-04-30T06:44:01.293ZWARNtest_unauthorized (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
81902025-04-30T06:44:01.298ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
81912025-04-30T06:44:01.298ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
81922025-04-30T06:44:01.299ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
81932025-04-30T06:44:01.307ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81942025-04-30T06:44:01.314ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
81952025-04-30T06:44:01.314ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
81962025-04-30T06:44:01.314ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
81972025-04-30T06:44:01.319ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
81982025-04-30T06:44:01.363ZWARNtest_unauthorized (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
81992025-04-30T06:44:01.363ZWARNtest_unauthorized (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
82002025-04-30T06:44:01.363ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 729.85566ms iteration = 26 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
82012025-04-30T06:44:01.363ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 27 reason = Signaled
82022025-04-30T06:44:01.363ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
82032025-04-30T06:44:01.363ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
82042025-04-30T06:44:01.363ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
82052025-04-30T06:44:01.363ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
82062025-04-30T06:44:01.364ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
82072025-04-30T06:44:01.366ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
82082025-04-30T06:44:01.385ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82092025-04-30T06:44:01.390ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
82102025-04-30T06:44:01.390ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
82112025-04-30T06:44:01.390ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
82122025-04-30T06:44:01.390ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
82132025-04-30T06:44:01.394ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1a8b2318-abf8-406d-a389-112ab91a0de8, 0)]
82142025-04-30T06:44:01.394ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
82152025-04-30T06:44:01.395ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: d1 77 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .w..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:63397 req_id = 177f70f2-447f-4b3a-99f9-20d7a29cb702
82162025-04-30T06:44:01.395ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 53623,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63397 req_id = 177f70f2-447f-4b3a-99f9-20d7a29cb702
82172025-04-30T06:44:01.395ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
82182025-04-30T06:44:01.395ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
82192025-04-30T06:44:01.396ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:63397 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 177f70f2-447f-4b3a-99f9-20d7a29cb702
82202025-04-30T06:44:01.396ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4306, tv_nsec: 195270827 } })
82212025-04-30T06:44:01.396ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
82222025-04-30T06:44:01.396ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 67 c4 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e g...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:55350 req_id = d6756469-efa1-4cf5-9e17-ceda33a41301
82232025-04-30T06:44:01.396ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 26564,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:55350 req_id = d6756469-efa1-4cf5-9e17-ceda33a41301
82242025-04-30T06:44:01.396ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
82252025-04-30T06:44:01.396ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:55350 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = d6756469-efa1-4cf5-9e17-ceda33a41301
82262025-04-30T06:44:01.483ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
82272025-04-30T06:44:01.483ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
82282025-04-30T06:44:01.483ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
82292025-04-30T06:44:01.483ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
82302025-04-30T06:44:01.483ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
82312025-04-30T06:44:01.485ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
82322025-04-30T06:44:01.497ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: f8 a0 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:55054 req_id = c11a37c2-8258-4825-9d63-3acb7d4fa6f5
82332025-04-30T06:44:01.497ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 63648,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:55054 req_id = c11a37c2-8258-4825-9d63-3acb7d4fa6f5
82342025-04-30T06:44:01.497ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
82352025-04-30T06:44:01.497ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 63648, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:55054 req_id = c11a37c2-8258-4825-9d63-3acb7d4fa6f5
82362025-04-30T06:44:01.501ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
82372025-04-30T06:44:01.501ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
82382025-04-30T06:44:01.501ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
82392025-04-30T06:44:01.502ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82402025-04-30T06:44:01.509ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
82412025-04-30T06:44:01.509ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
82422025-04-30T06:44:01.509ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
82432025-04-30T06:44:01.510ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
82442025-04-30T06:44:01.513ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "2cd40473-411e-4604-bd23-11b3b1039eba", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
82452025-04-30T06:44:01.514ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
82462025-04-30T06:44:01.514ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
82472025-04-30T06:44:01.514ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
82482025-04-30T06:44:01.515ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
82492025-04-30T06:44:01.516ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "af675db8-9523-4568-bca6-dac968f4b40b", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
82502025-04-30T06:44:01.517ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
82512025-04-30T06:44:01.519ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "35c0bcfa-ee82-48e1-9a89-24548ed3751a", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
82522025-04-30T06:44:01.519ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
82532025-04-30T06:44:01.521ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "330a7b22-8c8e-4713-87ad-fd516cf974cd", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
82542025-04-30T06:44:01.521ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
82552025-04-30T06:44:01.537ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e16ab0e9-6d1c-46e2-a8ca-d7c8134ea1fc", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
82562025-04-30T06:44:01.537ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
82572025-04-30T06:44:01.642ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
82582025-04-30T06:44:01.645ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "39200a1e-2d23-4b43-b21e-ea4547686356", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
82592025-04-30T06:44:01.645ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
82602025-04-30T06:44:01.664ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "05ce3ccc-5bf0-48c1-ac62-07a476510c57", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
82612025-04-30T06:44:01.664ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
82622025-04-30T06:44:01.666ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1c42b466-5045-442c-bf55-a90f1199f985", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
82632025-04-30T06:44:01.666ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
82642025-04-30T06:44:01.676ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fda1a1cb-203e-44b8-8a41-3b6aa83b7658", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
82652025-04-30T06:44:01.676ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
82662025-04-30T06:44:01.678ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "60ff7e8a-acd0-4ae5-b67d-62f5998801cd", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
82672025-04-30T06:44:01.678ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
82682025-04-30T06:44:01.679ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "75cb8634-6482-45c7-a4e7-1b010ca22da9", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
82692025-04-30T06:44:01.679ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
82702025-04-30T06:44:01.681ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ec04576e-a600-44b4-9e14-4dcf48c271d0", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
82712025-04-30T06:44:01.681ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
82722025-04-30T06:44:01.682ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "87f8080f-d91d-46fc-8036-91f701f62f4d", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
82732025-04-30T06:44:01.682ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
82742025-04-30T06:44:01.684ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5a6df9a4-20a4-46b3-85ce-da3ed157c515", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
82752025-04-30T06:44:01.684ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
82762025-04-30T06:44:01.686ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4f40a537-9e5e-4957-b294-dab6d7a2b69b", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
82772025-04-30T06:44:01.686ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
82782025-04-30T06:44:01.688ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f0cff501-11e8-48d4-abaf-c904d215f7ae", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
82792025-04-30T06:44:01.689ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
82802025-04-30T06:44:01.691ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "58da04c8-bd5d-422f-b036-9f9987e7e821", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
82812025-04-30T06:44:01.691ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
82822025-04-30T06:44:01.693ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "129f2eed-c900-46f9-9199-12ac2c92e37c", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
82832025-04-30T06:44:01.693ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
82842025-04-30T06:44:01.693ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
82852025-04-30T06:44:01.693ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
82862025-04-30T06:44:01.693ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
82872025-04-30T06:44:01.693ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
82882025-04-30T06:44:01.694ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
82892025-04-30T06:44:01.698ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82902025-04-30T06:44:01.711ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
82912025-04-30T06:44:01.711ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
82922025-04-30T06:44:01.711ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
82932025-04-30T06:44:01.712ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
82942025-04-30T06:44:01.713ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e53e140b-56ce-463f-aa05-6c80e30934f6", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
82952025-04-30T06:44:01.713ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
82962025-04-30T06:44:01.714ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
82972025-04-30T06:44:01.714ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
82982025-04-30T06:44:01.715ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
82992025-04-30T06:44:01.717ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "250b34ff-92df-4f37-952e-f07fc0a6a73f", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
83002025-04-30T06:44:01.717ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
83012025-04-30T06:44:01.718ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ce718b1e-0ecd-4a69-a783-a16bfd9c9fb0", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
83022025-04-30T06:44:01.718ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
83032025-04-30T06:44:01.721ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "747bd958-63ca-4813-8893-875f8a0440ea", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
83042025-04-30T06:44:01.721ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
83052025-04-30T06:44:01.723ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9e7e87ff-7843-4880-8e5b-e02e202e7b04", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
83062025-04-30T06:44:01.723ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
83072025-04-30T06:44:01.726ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "021a62a6-cee1-4a88-80c8-e58b2ae9ae72", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
83082025-04-30T06:44:01.726ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
83092025-04-30T06:44:01.731ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e77b3fc7-8be8-4400-8712-4471fbb55243", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
83102025-04-30T06:44:01.731ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
83112025-04-30T06:44:01.735ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5a2dcd08-6476-4faa-893c-9a4cd9afde3d", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
83122025-04-30T06:44:01.735ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
83132025-04-30T06:44:01.739ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7212919c-636b-48ac-b736-512d6946d8c1", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
83142025-04-30T06:44:01.745ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
83152025-04-30T06:44:01.750ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b45305e5-f125-4bec-969c-92e29c5b86bb", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
83162025-04-30T06:44:01.750ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
83172025-04-30T06:44:01.753ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
83182025-04-30T06:44:01.756ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "51b7cd2a-69d4-447b-b2c3-fcee09e5e5a3", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
83192025-04-30T06:44:01.756ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
83202025-04-30T06:44:01.763ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "84317a26-41b9-40f6-a10c-9d1748fad0d2", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
83212025-04-30T06:44:01.763ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
83222025-04-30T06:44:01.764ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8188ffdd-9084-4d90-a9cc-f2e6bfc81931", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
83232025-04-30T06:44:01.764ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
83242025-04-30T06:44:01.768ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "54d7097f-526f-4b56-ac99-364fd1eae9b2", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
83252025-04-30T06:44:01.768ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
83262025-04-30T06:44:01.774ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "af1bb980-4e3b-4ba5-a630-9d87ccfd897c", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
83272025-04-30T06:44:01.774ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
83282025-04-30T06:44:01.777ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "778e855f-7a49-44ad-b0c7-fb7756ba16f1", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
83292025-04-30T06:44:01.777ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
83302025-04-30T06:44:01.782ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f2d6b102-1b99-4c38-972f-6e7f545e7f93", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
83312025-04-30T06:44:01.783ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
83322025-04-30T06:44:01.786ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b9c3d873-ab6d-4e10-91b8-70883862eda6", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
83332025-04-30T06:44:01.787ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
83342025-04-30T06:44:01.790ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "55b86329-0ee2-4aff-958f-e5da122ebfff", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
83352025-04-30T06:44:01.790ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
83362025-04-30T06:44:01.793ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a1a7a79c-3cc5-4aaa-a531-f47e0ecbec68", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
83372025-04-30T06:44:01.793ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
83382025-04-30T06:44:01.797ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5068ca01-4491-4c3d-9b80-88eca0b2e631", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
83392025-04-30T06:44:01.797ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
83402025-04-30T06:44:01.801ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c6a27961-ba22-479f-b673-3ee095ebf141", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
83412025-04-30T06:44:01.802ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
83422025-04-30T06:44:01.805ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
83432025-04-30T06:44:01.805ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
83442025-04-30T06:44:01.805ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
83452025-04-30T06:44:01.805ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
83462025-04-30T06:44:01.806ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
83472025-04-30T06:44:01.807ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9e7b3fba-54d7-4de9-8a63-728a04054d3e", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
83482025-04-30T06:44:01.807ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
83492025-04-30T06:44:01.809ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "93e2ddda-b0b3-4580-9de9-3e4e104c2921", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
83502025-04-30T06:44:01.809ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
83512025-04-30T06:44:01.810ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83522025-04-30T06:44:01.817ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
83532025-04-30T06:44:01.817ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
83542025-04-30T06:44:01.818ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
83552025-04-30T06:44:01.818ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
83562025-04-30T06:44:01.824ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c7fa2ea1-41f0-41ee-a105-c4f648655a33", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
83572025-04-30T06:44:01.824ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
83582025-04-30T06:44:01.826ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
83592025-04-30T06:44:01.826ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
83602025-04-30T06:44:01.827ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
83612025-04-30T06:44:01.828ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3531a20a-966c-45a3-88e3-2c01fb87721a", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
83622025-04-30T06:44:01.828ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
83632025-04-30T06:44:01.830ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd7092dc-2091-40e4-b407-f229d7b2a0d9", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
83642025-04-30T06:44:01.830ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
83652025-04-30T06:44:01.830ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
83662025-04-30T06:44:01.830ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
83672025-04-30T06:44:01.830ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
83682025-04-30T06:44:01.831ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
83692025-04-30T06:44:01.834ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83702025-04-30T06:44:01.842ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
83712025-04-30T06:44:01.843ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
83722025-04-30T06:44:01.843ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
83732025-04-30T06:44:01.843ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
83742025-04-30T06:44:01.843ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
83752025-04-30T06:44:01.843ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
83762025-04-30T06:44:01.849ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83772025-04-30T06:44:01.856ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
83782025-04-30T06:44:01.856ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
83792025-04-30T06:44:01.856ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
83802025-04-30T06:44:01.856ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
83812025-04-30T06:44:01.935ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
83822025-04-30T06:44:01.935ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory
83832025-04-30T06:44:01.937ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:36749 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
83842025-04-30T06:44:01.938ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:36749 req_id = 7b6932cc-0e05-494a-b9c3-1b63596be154 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
83852025-04-30T06:44:01.939ZINFOtest_unauthorized (dropshot): request completed latency_us = 695 local_addr = [::1]:57786 method = GET remote_addr = [::1]:36749 req_id = 7b6932cc-0e05-494a-b9c3-1b63596be154 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
83862025-04-30T06:44:01.940ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = http://[::1]:57786 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:57786/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "7b6932cc-0e05-494a-b9c3-1b63596be154", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
83872025-04-30T06:44:01.943ZDEBGtest_unauthorized (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
83882025-04-30T06:44:01.943ZDEBGtest_unauthorized (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
83892025-04-30T06:44:01.943ZDEBGtest_unauthorized (BackgroundTasks): finished collection background_task = inventory_collection
83902025-04-30T06:44:01.947ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
83912025-04-30T06:44:01.948ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
83922025-04-30T06:44:01.948ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
83932025-04-30T06:44:01.948ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
83942025-04-30T06:44:01.952ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
83952025-04-30T06:44:01.979ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83962025-04-30T06:44:01.989ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
83972025-04-30T06:44:01.989ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
83982025-04-30T06:44:01.989ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
83992025-04-30T06:44:01.990ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
84002025-04-30T06:44:02.001ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
84012025-04-30T06:44:02.036ZWARNtest_unauthorized (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
84022025-04-30T06:44:02.036ZWARNtest_unauthorized (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
84032025-04-30T06:44:02.036ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 672.838426ms iteration = 27 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
84042025-04-30T06:44:02.036ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 28 reason = Signaled
84052025-04-30T06:44:02.036ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
84062025-04-30T06:44:02.036ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
84072025-04-30T06:44:02.036ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
84082025-04-30T06:44:02.037ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
84092025-04-30T06:44:02.038ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
84102025-04-30T06:44:02.040ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84112025-04-30T06:44:02.050ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
84122025-04-30T06:44:02.050ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
84132025-04-30T06:44:02.050ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
84142025-04-30T06:44:02.051ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
84152025-04-30T06:44:02.055ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
84162025-04-30T06:44:02.055ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
84172025-04-30T06:44:02.055ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
84182025-04-30T06:44:02.055ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
84192025-04-30T06:44:02.056ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
84202025-04-30T06:44:02.056ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1a8b2318-abf8-406d-a389-112ab91a0de8, 0)]
84212025-04-30T06:44:02.056ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
84222025-04-30T06:44:02.065ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: d1 db 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:55950 req_id = 58c2b5db-df0f-45d4-9d16-a6064b8d3ad8
84232025-04-30T06:44:02.065ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 53723,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:55950 req_id = 58c2b5db-df0f-45d4-9d16-a6064b8d3ad8
84242025-04-30T06:44:02.065ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
84252025-04-30T06:44:02.065ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
84262025-04-30T06:44:02.065ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:55950 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 58c2b5db-df0f-45d4-9d16-a6064b8d3ad8
84272025-04-30T06:44:02.066ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84282025-04-30T06:44:02.071ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
84292025-04-30T06:44:02.071ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
84302025-04-30T06:44:02.071ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
84312025-04-30T06:44:02.072ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
84322025-04-30T06:44:02.072ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4306, tv_nsec: 871516812 } })
84332025-04-30T06:44:02.072ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
84342025-04-30T06:44:02.073ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: d6 1b 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:51320 req_id = f74c52c1-6b7f-462f-8c8b-629bc7fb1bd4
84352025-04-30T06:44:02.073ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 54811,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51320 req_id = f74c52c1-6b7f-462f-8c8b-629bc7fb1bd4
84362025-04-30T06:44:02.073ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
84372025-04-30T06:44:02.074ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:51320 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = f74c52c1-6b7f-462f-8c8b-629bc7fb1bd4
84382025-04-30T06:44:02.141ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
84392025-04-30T06:44:02.141ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
84402025-04-30T06:44:02.142ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
84412025-04-30T06:44:02.143ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
84422025-04-30T06:44:02.143ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 92 dd 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:57671 req_id = 1a52b904-4518-4cad-b641-763277576619
84432025-04-30T06:44:02.143ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 37597,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:57671 req_id = 1a52b904-4518-4cad-b641-763277576619
84442025-04-30T06:44:02.144ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
84452025-04-30T06:44:02.144ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 37597, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:57671 req_id = 1a52b904-4518-4cad-b641-763277576619
84462025-04-30T06:44:02.144ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
84472025-04-30T06:44:02.144ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
84482025-04-30T06:44:02.144ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
84492025-04-30T06:44:02.146ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "bc7e7656-15b1-473a-96df-7e8654b67c3f", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
84502025-04-30T06:44:02.147ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
84512025-04-30T06:44:02.148ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ed7fbd75-bbc0-4199-bfc3-02531ab2b7c0", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
84522025-04-30T06:44:02.148ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
84532025-04-30T06:44:02.153ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "16e04a5a-3223-4b09-b83c-b956a8439a0b", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
84542025-04-30T06:44:02.153ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
84552025-04-30T06:44:02.155ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "397996ca-7798-4176-9edb-bda28a224de6", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
84562025-04-30T06:44:02.155ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
84572025-04-30T06:44:02.159ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "857c20fa-9f72-475b-8d43-17c7500d61a8", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
84582025-04-30T06:44:02.159ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
84592025-04-30T06:44:02.161ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3aa31a3f-3937-4c60-b453-054ad6d6aaf6", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
84602025-04-30T06:44:02.161ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
84612025-04-30T06:44:02.163ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ba531df2-91ac-4753-beec-cc4bb4435448", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
84622025-04-30T06:44:02.163ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
84632025-04-30T06:44:02.168ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f5f54752-864d-47b8-b632-c24472b63484", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
84642025-04-30T06:44:02.169ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
84652025-04-30T06:44:02.171ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "123eed17-b243-4f8d-bfe2-cf3f046e7bcf", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
84662025-04-30T06:44:02.171ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
84672025-04-30T06:44:02.172ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "645052ad-a08a-49d7-99e9-24c6d72300a3", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
84682025-04-30T06:44:02.173ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
84692025-04-30T06:44:02.173ZDEBGtest_unauthorized (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
84702025-04-30T06:44:02.173ZDEBGtest_unauthorized (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
84712025-04-30T06:44:02.173ZDEBGtest_unauthorized (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a
84722025-04-30T06:44:02.174ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
84732025-04-30T06:44:02.175ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d87abafe-5f0d-43dd-bfc8-6e297867dfbf", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
84742025-04-30T06:44:02.175ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
84752025-04-30T06:44:02.177ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9c41f9ea-d51f-413f-93ae-eea3e5c861d5", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
84762025-04-30T06:44:02.177ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
84772025-04-30T06:44:02.179ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 2 table_name = fields_string
84782025-04-30T06:44:02.179ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
84792025-04-30T06:44:02.181ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9525eaf8-7ce5-41d2-8a25-22892c2d513e", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
84802025-04-30T06:44:02.181ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
84812025-04-30T06:44:02.182ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
84822025-04-30T06:44:02.183ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 1 table_name = measurements_i64
84832025-04-30T06:44:02.183ZTRACtest_unauthorized (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
84842025-04-30T06:44:02.185ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b18ad75a-04a5-4d2b-9560-180282a7e26c", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
84852025-04-30T06:44:02.185ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
84862025-04-30T06:44:02.187ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "71de3b52-26f2-40de-b7aa-394407eef843", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
84872025-04-30T06:44:02.187ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
84882025-04-30T06:44:02.189ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "41f4b1ce-e18f-49ce-a27f-70c904330e5f", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
84892025-04-30T06:44:02.189ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
84902025-04-30T06:44:02.191ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d82e074b-e1b3-4608-a10d-22d886e5e76a", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
84912025-04-30T06:44:02.191ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
84922025-04-30T06:44:02.196ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "81c436ea-da64-4611-be06-4a05715a16dd", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
84932025-04-30T06:44:02.196ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
84942025-04-30T06:44:02.198ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "36a74a6d-ecbc-4009-a8ed-0fbcf3b1dad7", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
84952025-04-30T06:44:02.198ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
84962025-04-30T06:44:02.200ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "30d6e6a4-0f17-4cdb-a6dc-9035d76013a4", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
84972025-04-30T06:44:02.200ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
84982025-04-30T06:44:02.200ZDEBGtest_unauthorized (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
84992025-04-30T06:44:02.200ZDEBGtest_unauthorized (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
85002025-04-30T06:44:02.202ZTRACtest_unauthorized (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
85012025-04-30T06:44:02.202ZDEBGtest_unauthorized (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
85022025-04-30T06:44:02.202ZTRACtest_unauthorized (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
85032025-04-30T06:44:02.203ZTRACtest_unauthorized (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
85042025-04-30T06:44:02.203ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b589f21a-7b3e-4f42-8cd7-90cbb5ce6eb1", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
85052025-04-30T06:44:02.203ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
85062025-04-30T06:44:02.204ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b8af83f0-7337-4024-ba0b-8dc8976909a2", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
85072025-04-30T06:44:02.204ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
85082025-04-30T06:44:02.205ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "abf8eb31-00d1-49f9-a3bd-1aea54fe60e0", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
85092025-04-30T06:44:02.205ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
85102025-04-30T06:44:02.210ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "36d33d82-afad-4f46-a255-588e2c1e4a8c", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
85112025-04-30T06:44:02.210ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
85122025-04-30T06:44:02.212ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "30dee058-f4d2-46b2-b177-158116ef3703", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
85132025-04-30T06:44:02.212ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
85142025-04-30T06:44:02.222ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0d07812b-721b-4b83-9b21-21d97bbdaae0", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
85152025-04-30T06:44:02.223ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
85162025-04-30T06:44:02.224ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "668b4e64-4183-4603-b288-aef63190615c", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
85172025-04-30T06:44:02.224ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
85182025-04-30T06:44:02.231ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "194ace01-6b70-4ba8-8118-524752f40665", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
85192025-04-30T06:44:02.231ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
85202025-04-30T06:44:02.233ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
85212025-04-30T06:44:02.233ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
85222025-04-30T06:44:02.233ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
85232025-04-30T06:44:02.233ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
85242025-04-30T06:44:02.233ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
85252025-04-30T06:44:02.234ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d0d4c869-1f85-440e-9afe-b014b0373ff7", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
85262025-04-30T06:44:02.234ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
85272025-04-30T06:44:02.236ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f611c83f-545e-448d-ad00-0cd96a8acfb9", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
85282025-04-30T06:44:02.236ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
85292025-04-30T06:44:02.236ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85302025-04-30T06:44:02.265ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
85312025-04-30T06:44:02.265ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
85322025-04-30T06:44:02.265ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
85332025-04-30T06:44:02.266ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
85342025-04-30T06:44:02.268ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "af4667e8-b2fe-4301-8d89-117b5092b25f", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
85352025-04-30T06:44:02.268ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
85362025-04-30T06:44:02.270ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f1151029-d879-433e-b9b2-6be50848d5ad", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
85372025-04-30T06:44:02.271ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
85382025-04-30T06:44:02.271ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
85392025-04-30T06:44:02.271ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
85402025-04-30T06:44:02.286ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
85412025-04-30T06:44:02.287ZWARNtest_unauthorized (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
85422025-04-30T06:44:02.288ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "35f52bab-41f4-476c-9e10-83ebdbe70af1", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
85432025-04-30T06:44:02.288ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
85442025-04-30T06:44:02.289ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4da8fe07-9de5-423f-870e-015fe414c52f", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
85452025-04-30T06:44:02.289ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
85462025-04-30T06:44:02.290ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e3179779-d4ef-493d-9fd8-d138c757c93d", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
85472025-04-30T06:44:02.290ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
85482025-04-30T06:44:02.293ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9999ce85-20e9-4a05-8210-896ef992bfa7", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
85492025-04-30T06:44:02.293ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
85502025-04-30T06:44:02.295ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "dc5df7af-7e6f-4728-8769-92c62ec528b9", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
85512025-04-30T06:44:02.295ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
85522025-04-30T06:44:02.297ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0520a106-0ec0-42aa-8ff3-cfa1e61e48c8", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
85532025-04-30T06:44:02.297ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
85542025-04-30T06:44:02.305ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ce1ec404-3c4e-498c-b334-44c5b59c0ee2", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
85552025-04-30T06:44:02.305ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
85562025-04-30T06:44:02.309ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2f83ce9c-7073-482e-931f-c699a1beae07", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
85572025-04-30T06:44:02.309ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
85582025-04-30T06:44:02.309ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c69f269c-cd55-4cce-833f-960d6e41fb1c", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
85592025-04-30T06:44:02.309ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
85602025-04-30T06:44:02.314ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e967d4f4-bdd1-48d9-bb86-b1c2afcf86ea", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
85612025-04-30T06:44:02.314ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
85622025-04-30T06:44:02.319ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "99cf245e-7e02-4705-8534-2795dbe4e801", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
85632025-04-30T06:44:02.319ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
85642025-04-30T06:44:02.320ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d0390851-d18f-41d4-a779-3d7207694452", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
85652025-04-30T06:44:02.320ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
85662025-04-30T06:44:02.321ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "31189095-66a5-4d33-bad3-7bb15574a1f1", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
85672025-04-30T06:44:02.322ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
85682025-04-30T06:44:02.322ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
85692025-04-30T06:44:02.322ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
85702025-04-30T06:44:02.322ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
85712025-04-30T06:44:02.323ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
85722025-04-30T06:44:02.327ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
85732025-04-30T06:44:02.328ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85742025-04-30T06:44:02.340ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
85752025-04-30T06:44:02.340ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
85762025-04-30T06:44:02.340ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
85772025-04-30T06:44:02.340ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
85782025-04-30T06:44:02.340ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
85792025-04-30T06:44:02.340ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
85802025-04-30T06:44:02.351ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85812025-04-30T06:44:02.362ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
85822025-04-30T06:44:02.362ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
85832025-04-30T06:44:02.362ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
85842025-04-30T06:44:02.363ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
85852025-04-30T06:44:02.381ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
85862025-04-30T06:44:02.381ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
85872025-04-30T06:44:02.381ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
85882025-04-30T06:44:02.381ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
85892025-04-30T06:44:02.383ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
85902025-04-30T06:44:02.447ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
85912025-04-30T06:44:02.447ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory
85922025-04-30T06:44:02.447ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:45966 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
85932025-04-30T06:44:02.448ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:45966 req_id = ee570566-aac9-433e-a1ec-85e551a70823 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
85942025-04-30T06:44:02.448ZINFOtest_unauthorized (dropshot): request completed latency_us = 498 local_addr = [::1]:57786 method = GET remote_addr = [::1]:45966 req_id = ee570566-aac9-433e-a1ec-85e551a70823 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
85952025-04-30T06:44:02.449ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = http://[::1]:57786 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:57786/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "ee570566-aac9-433e-a1ec-85e551a70823", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
85962025-04-30T06:44:02.450ZDEBGtest_unauthorized (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
85972025-04-30T06:44:02.450ZDEBGtest_unauthorized (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
85982025-04-30T06:44:02.450ZDEBGtest_unauthorized (BackgroundTasks): finished collection background_task = inventory_collection
85992025-04-30T06:44:02.453ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
86002025-04-30T06:44:02.453ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
86012025-04-30T06:44:02.453ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
86022025-04-30T06:44:02.453ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
86032025-04-30T06:44:02.454ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
86042025-04-30T06:44:02.454ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86052025-04-30T06:44:02.460ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
86062025-04-30T06:44:02.460ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
86072025-04-30T06:44:02.460ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
86082025-04-30T06:44:02.460ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
86092025-04-30T06:44:02.464ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86102025-04-30T06:44:02.472ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
86112025-04-30T06:44:02.472ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
86122025-04-30T06:44:02.472ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
86132025-04-30T06:44:02.473ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
86142025-04-30T06:44:02.474ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
86152025-04-30T06:44:02.474ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
86162025-04-30T06:44:02.475ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
86172025-04-30T06:44:02.511ZWARNtest_unauthorized (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
86182025-04-30T06:44:02.511ZWARNtest_unauthorized (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
86192025-04-30T06:44:02.511ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 474.659208ms iteration = 28 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
86202025-04-30T06:44:02.511ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 29 reason = Signaled
86212025-04-30T06:44:02.511ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
86222025-04-30T06:44:02.511ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
86232025-04-30T06:44:02.511ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
86242025-04-30T06:44:02.511ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
86252025-04-30T06:44:02.512ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
86262025-04-30T06:44:02.514ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86272025-04-30T06:44:02.521ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
86282025-04-30T06:44:02.521ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
86292025-04-30T06:44:02.521ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
86302025-04-30T06:44:02.522ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
86312025-04-30T06:44:02.526ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1a8b2318-abf8-406d-a389-112ab91a0de8, 0)]
86322025-04-30T06:44:02.526ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
86332025-04-30T06:44:02.527ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 31 d7 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 1............_mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:54357 req_id = f1703ee4-4bf4-4e68-b6ec-fd92aee6fd30
86342025-04-30T06:44:02.527ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 12759,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54357 req_id = f1703ee4-4bf4-4e68-b6ec-fd92aee6fd30
86352025-04-30T06:44:02.527ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
86362025-04-30T06:44:02.527ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
86372025-04-30T06:44:02.527ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:54357 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = f1703ee4-4bf4-4e68-b6ec-fd92aee6fd30
86382025-04-30T06:44:02.528ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
86392025-04-30T06:44:02.528ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4307, tv_nsec: 327444428 } })
86402025-04-30T06:44:02.528ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
86412025-04-30T06:44:02.529ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: d4 c2 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:59135 req_id = d2d028f1-880f-45ec-8913-4ae6ebdd5884
86422025-04-30T06:44:02.529ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 54466,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:59135 req_id = d2d028f1-880f-45ec-8913-4ae6ebdd5884
86432025-04-30T06:44:02.530ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
86442025-04-30T06:44:02.530ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:59135 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = d2d028f1-880f-45ec-8913-4ae6ebdd5884
86452025-04-30T06:44:02.599ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
86462025-04-30T06:44:02.600ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
86472025-04-30T06:44:02.600ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
86482025-04-30T06:44:02.600ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
86492025-04-30T06:44:02.600ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
86502025-04-30T06:44:02.601ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
86512025-04-30T06:44:02.601ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 87 07 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:54822 req_id = f7dd1d4f-ef7c-46c8-b6fe-6384ddc2b327
86522025-04-30T06:44:02.603ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 34567,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54822 req_id = f7dd1d4f-ef7c-46c8-b6fe-6384ddc2b327
86532025-04-30T06:44:02.603ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
86542025-04-30T06:44:02.603ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 34567, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:54822 req_id = f7dd1d4f-ef7c-46c8-b6fe-6384ddc2b327
86552025-04-30T06:44:02.604ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
86562025-04-30T06:44:02.604ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
86572025-04-30T06:44:02.604ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
86582025-04-30T06:44:02.605ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86592025-04-30T06:44:02.610ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
86602025-04-30T06:44:02.610ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
86612025-04-30T06:44:02.610ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
86622025-04-30T06:44:02.611ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
86632025-04-30T06:44:02.625ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "e382d971-0ceb-468d-b648-b454683336b1", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
86642025-04-30T06:44:02.625ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
86652025-04-30T06:44:02.626ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
86662025-04-30T06:44:02.626ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
86672025-04-30T06:44:02.626ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
86682025-04-30T06:44:02.648ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d75ac582-bd35-43a2-bcc6-f2ff693f2032", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
86692025-04-30T06:44:02.651ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
86702025-04-30T06:44:02.673ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "37bcea79-dd2c-4515-9ad6-e6823eba9128", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
86712025-04-30T06:44:02.674ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
86722025-04-30T06:44:02.676ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "613d952b-3a48-4323-8776-35a53b86d95f", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
86732025-04-30T06:44:02.676ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
86742025-04-30T06:44:02.679ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f029a557-a06f-4d53-bf53-cb1f162c3131", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
86752025-04-30T06:44:02.679ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
86762025-04-30T06:44:02.682ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "92604561-b0d2-433c-9c53-42d1f27fea5c", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
86772025-04-30T06:44:02.682ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
86782025-04-30T06:44:02.684ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9ae1fa16-c88c-4ea0-bc38-964139cc92ab", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
86792025-04-30T06:44:02.684ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
86802025-04-30T06:44:02.687ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8e725125-5cbf-4a4f-9072-054e489a7f72", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
86812025-04-30T06:44:02.687ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
86822025-04-30T06:44:02.688ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
86832025-04-30T06:44:02.689ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "28062153-70a1-4f20-bcbe-d81637c0102a", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
86842025-04-30T06:44:02.690ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
86852025-04-30T06:44:02.691ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fe5ca6ac-c4c0-4012-9bd6-112cca4a4836", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
86862025-04-30T06:44:02.692ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
86872025-04-30T06:44:02.693ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7fc3857f-780f-47d3-a706-301e2d1db409", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
86882025-04-30T06:44:02.693ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
86892025-04-30T06:44:02.694ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e37986d1-cc86-4c88-b053-b938df92c92b", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
86902025-04-30T06:44:02.694ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
86912025-04-30T06:44:02.695ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2115d3f6-f34b-4d82-a4cc-87958254e053", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
86922025-04-30T06:44:02.695ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
86932025-04-30T06:44:02.697ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c3e3256a-f437-425f-8387-05b7cac0c88d", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
86942025-04-30T06:44:02.697ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
86952025-04-30T06:44:02.699ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0e808c28-c382-446e-b05f-5e03f6f8307b", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
86962025-04-30T06:44:02.699ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
86972025-04-30T06:44:02.701ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bc27cf55-e210-473a-ad3c-91f0ba0dba9e", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
86982025-04-30T06:44:02.701ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
86992025-04-30T06:44:02.703ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4f50e9f0-417e-477f-b3aa-ad71338c37da", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
87002025-04-30T06:44:02.703ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
87012025-04-30T06:44:02.705ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1f7bf5a8-f59a-489b-94ef-5d6e7144b606", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
87022025-04-30T06:44:02.705ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
87032025-04-30T06:44:02.707ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9f5277b2-e464-4df3-aae1-19ae8f7db4ba", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
87042025-04-30T06:44:02.708ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
87052025-04-30T06:44:02.708ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "24252a12-9a19-4f0b-896d-b4ab8ea7cc82", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
87062025-04-30T06:44:02.708ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
87072025-04-30T06:44:02.711ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "41b8c034-1e19-4ad3-acb4-045a8195d08e", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
87082025-04-30T06:44:02.712ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
87092025-04-30T06:44:02.713ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "08e73e58-8630-4c1f-a29e-95750a5597b6", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
87102025-04-30T06:44:02.713ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
87112025-04-30T06:44:02.714ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "771f3c17-c6ee-42a7-8bab-a6f5897e64a5", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
87122025-04-30T06:44:02.714ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
87132025-04-30T06:44:02.715ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6847a2f0-7da4-40ef-838e-b04ac738ed58", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
87142025-04-30T06:44:02.716ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
87152025-04-30T06:44:02.718ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d7fed8ee-c375-4c60-b5fb-eea0a4e0ba65", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
87162025-04-30T06:44:02.718ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
87172025-04-30T06:44:02.720ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "26a8863c-9d2b-43c8-baa1-3447382d942c", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
87182025-04-30T06:44:02.720ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
87192025-04-30T06:44:02.731ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "66fcaf60-732a-4e53-9154-28f968b530f2", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
87202025-04-30T06:44:02.731ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
87212025-04-30T06:44:02.734ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "13f3be20-c4c1-4839-b2f3-1035803a875f", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
87222025-04-30T06:44:02.734ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
87232025-04-30T06:44:02.736ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "eb473bbf-d585-41ac-994d-05eb59e1707d", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
87242025-04-30T06:44:02.736ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
87252025-04-30T06:44:02.738ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7c96c6df-795a-4141-81e4-8316ef50a29d", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
87262025-04-30T06:44:02.738ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
87272025-04-30T06:44:02.739ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
87282025-04-30T06:44:02.739ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
87292025-04-30T06:44:02.739ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
87302025-04-30T06:44:02.739ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
87312025-04-30T06:44:02.740ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
87322025-04-30T06:44:02.740ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b1030d5a-7c20-4f3e-b2d5-a92af9a50b7c", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
87332025-04-30T06:44:02.740ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
87342025-04-30T06:44:02.742ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a7229faa-17a0-4db2-a44a-d879aa569636", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
87352025-04-30T06:44:02.742ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
87362025-04-30T06:44:02.743ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87372025-04-30T06:44:02.748ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
87382025-04-30T06:44:02.749ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
87392025-04-30T06:44:02.749ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
87402025-04-30T06:44:02.749ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
87412025-04-30T06:44:02.749ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "69df0710-b637-471f-8a06-d7d9701a14cf", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
87422025-04-30T06:44:02.749ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
87432025-04-30T06:44:02.751ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
87442025-04-30T06:44:02.751ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
87452025-04-30T06:44:02.752ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
87462025-04-30T06:44:02.752ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c860294b-96b7-4a14-bed8-d96ce8a09331", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
87472025-04-30T06:44:02.752ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
87482025-04-30T06:44:02.756ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9c1271bf-cd40-4360-bc29-a04016e26997", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
87492025-04-30T06:44:02.756ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
87502025-04-30T06:44:02.760ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "010ca99a-9f93-4ec4-bbe6-2248d30ed165", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
87512025-04-30T06:44:02.760ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
87522025-04-30T06:44:02.763ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6371fc92-a430-4633-822d-45d2cdc892b5", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
87532025-04-30T06:44:02.763ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
87542025-04-30T06:44:02.767ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "78e1aed5-988e-4d61-82ce-9c62e942dd16", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
87552025-04-30T06:44:02.767ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
87562025-04-30T06:44:02.769ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "935968f3-202b-462f-b92d-a7587d9cd98a", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
87572025-04-30T06:44:02.769ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
87582025-04-30T06:44:02.772ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0445ac2f-03e8-40b0-a7ba-73970fa22683", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
87592025-04-30T06:44:02.772ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
87602025-04-30T06:44:02.774ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "00d6ffe4-d7d4-45f5-bb39-4796fb04af54", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
87612025-04-30T06:44:02.774ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
87622025-04-30T06:44:02.775ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "340639ce-28cb-40e7-8de3-903ee1f3be43", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
87632025-04-30T06:44:02.775ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
87642025-04-30T06:44:02.776ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "290458f4-b824-4137-9356-48a42b3e8306", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
87652025-04-30T06:44:02.776ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
87662025-04-30T06:44:02.777ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4e62974a-cba5-401d-9366-bc4436895cd4", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
87672025-04-30T06:44:02.777ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
87682025-04-30T06:44:02.779ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "dbb64afa-79cd-4de9-ab56-580904169f80", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
87692025-04-30T06:44:02.779ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
87702025-04-30T06:44:02.780ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
87712025-04-30T06:44:02.780ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
87722025-04-30T06:44:02.780ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
87732025-04-30T06:44:02.780ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
87742025-04-30T06:44:02.783ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87752025-04-30T06:44:02.789ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
87762025-04-30T06:44:02.790ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
87772025-04-30T06:44:02.790ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
87782025-04-30T06:44:02.790ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
87792025-04-30T06:44:02.790ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
87802025-04-30T06:44:02.791ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
87812025-04-30T06:44:02.795ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87822025-04-30T06:44:02.807ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
87832025-04-30T06:44:02.807ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
87842025-04-30T06:44:02.807ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
87852025-04-30T06:44:02.807ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
87862025-04-30T06:44:02.809ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
87872025-04-30T06:44:02.883ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
87882025-04-30T06:44:02.883ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory
87892025-04-30T06:44:02.884ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
87902025-04-30T06:44:02.884ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
87912025-04-30T06:44:02.884ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
87922025-04-30T06:44:02.884ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
87932025-04-30T06:44:02.885ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
87942025-04-30T06:44:02.885ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:40366 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
87952025-04-30T06:44:02.887ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:40366 req_id = 3399072b-236a-4d5d-852e-294d44b5bae5 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
87962025-04-30T06:44:02.888ZINFOtest_unauthorized (dropshot): request completed latency_us = 622 local_addr = [::1]:57786 method = GET remote_addr = [::1]:40366 req_id = 3399072b-236a-4d5d-852e-294d44b5bae5 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
87972025-04-30T06:44:02.888ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = http://[::1]:57786 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:57786/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "3399072b-236a-4d5d-852e-294d44b5bae5", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
87982025-04-30T06:44:02.889ZDEBGtest_unauthorized (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
87992025-04-30T06:44:02.890ZDEBGtest_unauthorized (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
88002025-04-30T06:44:02.890ZDEBGtest_unauthorized (BackgroundTasks): finished collection background_task = inventory_collection
88012025-04-30T06:44:02.892ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
88022025-04-30T06:44:02.892ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
88032025-04-30T06:44:02.892ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
88042025-04-30T06:44:02.892ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
88052025-04-30T06:44:02.893ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
88062025-04-30T06:44:02.893ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88072025-04-30T06:44:02.900ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
88082025-04-30T06:44:02.900ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
88092025-04-30T06:44:02.900ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
88102025-04-30T06:44:02.901ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
88112025-04-30T06:44:02.903ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88122025-04-30T06:44:02.909ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
88132025-04-30T06:44:02.909ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
88142025-04-30T06:44:02.910ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
88152025-04-30T06:44:02.910ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
88162025-04-30T06:44:02.912ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
88172025-04-30T06:44:02.913ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
88182025-04-30T06:44:02.914ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
88192025-04-30T06:44:02.956ZWARNtest_unauthorized (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
88202025-04-30T06:44:02.956ZWARNtest_unauthorized (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
88212025-04-30T06:44:02.957ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 445.66619ms iteration = 29 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
88222025-04-30T06:44:02.957ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 30 reason = Signaled
88232025-04-30T06:44:02.957ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
88242025-04-30T06:44:02.957ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
88252025-04-30T06:44:02.957ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
88262025-04-30T06:44:02.957ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
88272025-04-30T06:44:02.958ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
88282025-04-30T06:44:02.964ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
88292025-04-30T06:44:02.964ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88302025-04-30T06:44:02.970ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
88312025-04-30T06:44:02.970ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
88322025-04-30T06:44:02.970ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
88332025-04-30T06:44:02.970ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
88342025-04-30T06:44:02.973ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1a8b2318-abf8-406d-a389-112ab91a0de8, 0)]
88352025-04-30T06:44:02.973ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
88362025-04-30T06:44:02.974ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 48 56 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 HV..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:54719 req_id = fc610072-75ad-4a33-8c51-c885150530a8
88372025-04-30T06:44:02.974ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 18518,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54719 req_id = fc610072-75ad-4a33-8c51-c885150530a8
88382025-04-30T06:44:02.975ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
88392025-04-30T06:44:02.975ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
88402025-04-30T06:44:02.975ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:54719 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = fc610072-75ad-4a33-8c51-c885150530a8
88412025-04-30T06:44:02.975ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4307, tv_nsec: 774454095 } })
88422025-04-30T06:44:02.975ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
88432025-04-30T06:44:02.975ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 88 c1 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:65278 req_id = be959471-962e-4a67-b92a-e51687c5632f
88442025-04-30T06:44:02.975ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 35009,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:65278 req_id = be959471-962e-4a67-b92a-e51687c5632f
88452025-04-30T06:44:02.976ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
88462025-04-30T06:44:02.976ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:65278 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = be959471-962e-4a67-b92a-e51687c5632f
88472025-04-30T06:44:03.032ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
88482025-04-30T06:44:03.033ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
88492025-04-30T06:44:03.033ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
88502025-04-30T06:44:03.033ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
88512025-04-30T06:44:03.033ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
88522025-04-30T06:44:03.034ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
88532025-04-30T06:44:03.035ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: e8 23 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c .#..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:62080 req_id = 6492e1eb-be8c-4962-a147-273b410056c2
88542025-04-30T06:44:03.035ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 59427,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62080 req_id = 6492e1eb-be8c-4962-a147-273b410056c2
88552025-04-30T06:44:03.035ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
88562025-04-30T06:44:03.036ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 59427, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:62080 req_id = 6492e1eb-be8c-4962-a147-273b410056c2
88572025-04-30T06:44:03.036ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
88582025-04-30T06:44:03.036ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
88592025-04-30T06:44:03.036ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
88602025-04-30T06:44:03.037ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88612025-04-30T06:44:03.044ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
88622025-04-30T06:44:03.044ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
88632025-04-30T06:44:03.044ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
88642025-04-30T06:44:03.044ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
88652025-04-30T06:44:03.049ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
88662025-04-30T06:44:03.050ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
88672025-04-30T06:44:03.050ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
88682025-04-30T06:44:03.051ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "f0efa159-93b9-4fad-adce-3aa743e7e9c7", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
88692025-04-30T06:44:03.051ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
88702025-04-30T06:44:03.056ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fc3345f9-98d6-4aa6-a217-5d4142ce4023", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
88712025-04-30T06:44:03.056ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
88722025-04-30T06:44:03.061ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3ad3584f-c102-47b2-90d1-fb450126409a", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
88732025-04-30T06:44:03.062ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
88742025-04-30T06:44:03.066ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e819dce3-4d50-480a-8424-27020787e69f", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
88752025-04-30T06:44:03.066ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
88762025-04-30T06:44:03.071ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d3d4c75f-7f32-48be-87f2-9102ac9011c2", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
88772025-04-30T06:44:03.071ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
88782025-04-30T06:44:03.075ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5c5e6d57-915e-4ff2-ad1c-3d84ec9ef410", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
88792025-04-30T06:44:03.076ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
88802025-04-30T06:44:03.078ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "83cb6568-50a0-4ef4-995e-f52daa1f707b", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
88812025-04-30T06:44:03.078ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
88822025-04-30T06:44:03.083ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
88832025-04-30T06:44:03.083ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8384697c-de7a-4a6d-8400-a189279b6a96", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
88842025-04-30T06:44:03.083ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
88852025-04-30T06:44:03.085ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4da0a598-d491-41a9-ad4c-13fc61e023be", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
88862025-04-30T06:44:03.086ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
88872025-04-30T06:44:03.087ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "08807276-c6e1-43a3-8607-28d41ed9dd48", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
88882025-04-30T06:44:03.087ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
88892025-04-30T06:44:03.088ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1dd3ddef-27a8-4662-9e6c-668aa009501c", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
88902025-04-30T06:44:03.088ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
88912025-04-30T06:44:03.089ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b9c6bd84-44b0-4e8d-aa93-4e1b163595f6", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
88922025-04-30T06:44:03.089ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
88932025-04-30T06:44:03.091ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "695f1fd7-7c48-4fe8-b308-f719742f531e", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
88942025-04-30T06:44:03.092ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
88952025-04-30T06:44:03.092ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a161e5db-97bf-4820-b8db-253f9b3107e9", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
88962025-04-30T06:44:03.093ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
88972025-04-30T06:44:03.095ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "99ac593c-832c-405f-b16f-0be93edc7a19", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
88982025-04-30T06:44:03.095ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
88992025-04-30T06:44:03.097ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0dec17d1-6182-4888-a7d2-749e664a09de", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
89002025-04-30T06:44:03.097ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
89012025-04-30T06:44:03.099ZINFOtest_unauthorized (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
89022025-04-30T06:44:03.099ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "750a0787-61a5-48b4-af19-d2bcba4095ea", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
89032025-04-30T06:44:03.099ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
89042025-04-30T06:44:03.100ZDEBGtest_unauthorized (oximeter): client request body = None method = GET uri = http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
89052025-04-30T06:44:03.100ZTRACtest_unauthorized (dropshot_internal): incoming request local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = f95a2c43-7eb1-4af7-8920-717189b1cc34 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
89062025-04-30T06:44:03.100ZTRACtest_unauthorized (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = f95a2c43-7eb1-4af7-8920-717189b1cc34 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
89072025-04-30T06:44:03.100ZDEBGtest_unauthorized (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = f95a2c43-7eb1-4af7-8920-717189b1cc34 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
89082025-04-30T06:44:03.101ZDEBGtest_unauthorized (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = f95a2c43-7eb1-4af7-8920-717189b1cc34 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
89092025-04-30T06:44:03.106ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bbc86e6b-87d8-497a-817c-dd91088c86c3", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
89102025-04-30T06:44:03.106ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
89112025-04-30T06:44:03.108ZINFOtest_unauthorized (dropshot_internal): request completed latency_us = 8508 local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = f95a2c43-7eb1-4af7-8920-717189b1cc34 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
89122025-04-30T06:44:03.109ZDEBGtest_unauthorized (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "f95a2c43-7eb1-4af7-8920-717189b1cc34", "content-length": "405", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
89132025-04-30T06:44:03.110ZDEBGtest_unauthorized (oximeter): client request body = None method = GET uri = http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
89142025-04-30T06:44:03.110ZTRACtest_unauthorized (dropshot_internal): incoming request local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = fc901697-ba93-4e56-a5aa-0593066a1554 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
89152025-04-30T06:44:03.110ZTRACtest_unauthorized (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = fc901697-ba93-4e56-a5aa-0593066a1554 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
89162025-04-30T06:44:03.110ZDEBGtest_unauthorized (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = fc901697-ba93-4e56-a5aa-0593066a1554 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
89172025-04-30T06:44:03.111ZDEBGtest_unauthorized (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = fc901697-ba93-4e56-a5aa-0593066a1554 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
89182025-04-30T06:44:03.114ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7817ca06-b325-4ef4-8de4-ca770d950d8b", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
89192025-04-30T06:44:03.114ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
89202025-04-30T06:44:03.115ZINFOtest_unauthorized (dropshot_internal): request completed latency_us = 5227 local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = fc901697-ba93-4e56-a5aa-0593066a1554 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
89212025-04-30T06:44:03.124ZDEBGtest_unauthorized (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "fc901697-ba93-4e56-a5aa-0593066a1554", "content-length": "29", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
89222025-04-30T06:44:03.124ZTRACtest_unauthorized (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 8d752adb-08b9-405f-a11b-66019bf447b5
89232025-04-30T06:44:03.124ZTRACtest_unauthorized (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
89242025-04-30T06:44:03.124ZINFOtest_unauthorized (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
89252025-04-30T06:44:03.125ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5f8142d5-af87-44c0-a3f8-9d82fb6597aa", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
89262025-04-30T06:44:03.125ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
89272025-04-30T06:44:03.130ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "93d7ed91-d69f-4ed9-82c9-22e5142eaf56", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
89282025-04-30T06:44:03.130ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
89292025-04-30T06:44:03.131ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "47fb120a-779a-4479-9a8c-4e996dcd8e25", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
89302025-04-30T06:44:03.132ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
89312025-04-30T06:44:03.133ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6510e8cd-fb95-4c61-ab48-16f9442dce87", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
89322025-04-30T06:44:03.133ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
89332025-04-30T06:44:03.134ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
89342025-04-30T06:44:03.134ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
89352025-04-30T06:44:03.134ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
89362025-04-30T06:44:03.134ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
89372025-04-30T06:44:03.135ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
89382025-04-30T06:44:03.139ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "968eac88-d31a-4535-a475-25de080ccc12", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
89392025-04-30T06:44:03.139ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
89402025-04-30T06:44:03.139ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89412025-04-30T06:44:03.146ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
89422025-04-30T06:44:03.146ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
89432025-04-30T06:44:03.146ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
89442025-04-30T06:44:03.149ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
89452025-04-30T06:44:03.151ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6779e83d-a136-4079-82b2-23399112a5d4", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
89462025-04-30T06:44:03.151ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
89472025-04-30T06:44:03.151ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
89482025-04-30T06:44:03.151ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
89492025-04-30T06:44:03.152ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
89502025-04-30T06:44:03.157ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c46c2424-adac-4b21-8a79-3aff9e25f023", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
89512025-04-30T06:44:03.157ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
89522025-04-30T06:44:03.159ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6b52bd4a-fc68-4d57-920a-269087b6c433", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
89532025-04-30T06:44:03.160ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
89542025-04-30T06:44:03.164ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f22246a5-508f-43a4-8583-b2008b3a565e", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
89552025-04-30T06:44:03.164ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
89562025-04-30T06:44:03.167ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "efc074f1-8060-49e1-95dc-c0aeb495450f", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
89572025-04-30T06:44:03.174ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
89582025-04-30T06:44:03.175ZDEBGtest_unauthorized (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
89592025-04-30T06:44:03.175ZDEBGtest_unauthorized (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
89602025-04-30T06:44:03.175ZDEBGtest_unauthorized (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a
89612025-04-30T06:44:03.177ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
89622025-04-30T06:44:03.182ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3d047bf8-5e16-4a87-b94c-2bbc6d342d42", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
89632025-04-30T06:44:03.187ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
89642025-04-30T06:44:03.187ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 2 table_name = fields_string
89652025-04-30T06:44:03.188ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
89662025-04-30T06:44:03.188ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e86e1185-f172-4a08-9e39-533dbb27f9d3", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
89672025-04-30T06:44:03.188ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
89682025-04-30T06:44:03.190ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 1 table_name = measurements_i64
89692025-04-30T06:44:03.190ZTRACtest_unauthorized (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
89702025-04-30T06:44:03.190ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "75d734c8-498d-48ac-bd20-d152a2be9f50", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
89712025-04-30T06:44:03.191ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
89722025-04-30T06:44:03.193ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6c3f00b3-8200-4f3f-9ce0-211f9b8a1ceb", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
89732025-04-30T06:44:03.193ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
89742025-04-30T06:44:03.195ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ff80cbad-295d-4dc8-b258-790d28b105d4", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
89752025-04-30T06:44:03.195ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
89762025-04-30T06:44:03.197ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "880a7787-a62d-4f2b-889c-69bf61e2275b", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
89772025-04-30T06:44:03.197ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
89782025-04-30T06:44:03.200ZDEBGtest_unauthorized (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
89792025-04-30T06:44:03.200ZDEBGtest_unauthorized (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
89802025-04-30T06:44:03.207ZTRACtest_unauthorized (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
89812025-04-30T06:44:03.208ZDEBGtest_unauthorized (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
89822025-04-30T06:44:03.208ZTRACtest_unauthorized (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
89832025-04-30T06:44:03.208ZTRACtest_unauthorized (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
89842025-04-30T06:44:03.209ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d0d88214-62c7-48dd-8880-d56a2717669e", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
89852025-04-30T06:44:03.209ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
89862025-04-30T06:44:03.213ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
89872025-04-30T06:44:03.215ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "99031f90-2333-4f0c-929c-4e4fca1f7601", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
89882025-04-30T06:44:03.217ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
89892025-04-30T06:44:03.217ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "43c38765-2f7e-4d63-9c0b-213843fff484", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
89902025-04-30T06:44:03.217ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
89912025-04-30T06:44:03.218ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "88c4a949-523d-41b3-a0ff-4cbf378134a6", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
89922025-04-30T06:44:03.218ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
89932025-04-30T06:44:03.225ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b6099fc6-83f3-4e89-ae5b-97501e15b84e", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
89942025-04-30T06:44:03.225ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
89952025-04-30T06:44:03.227ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "37dd9477-0ca1-4503-96a9-1d80e9309a10", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
89962025-04-30T06:44:03.228ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
89972025-04-30T06:44:03.229ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "95a74b1b-6321-41aa-8146-28cdbd12032c", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
89982025-04-30T06:44:03.229ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
89992025-04-30T06:44:03.230ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "86b50d9d-0f42-4241-a8bb-6672ca3e25cb", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
90002025-04-30T06:44:03.231ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
90012025-04-30T06:44:03.232ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b86ee647-ec2f-4ab7-86a7-eda072c4a05e", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
90022025-04-30T06:44:03.232ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
90032025-04-30T06:44:03.233ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f7e5b5a7-6925-40ce-9d0b-bd1897755268", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
90042025-04-30T06:44:03.233ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
90052025-04-30T06:44:03.233ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
90062025-04-30T06:44:03.233ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
90072025-04-30T06:44:03.234ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
90082025-04-30T06:44:03.235ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
90092025-04-30T06:44:03.241ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90102025-04-30T06:44:03.255ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
90112025-04-30T06:44:03.255ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
90122025-04-30T06:44:03.255ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
90132025-04-30T06:44:03.255ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
90142025-04-30T06:44:03.255ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
90152025-04-30T06:44:03.256ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
90162025-04-30T06:44:03.272ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
90172025-04-30T06:44:03.302ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
90182025-04-30T06:44:03.302ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
90192025-04-30T06:44:03.302ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
90202025-04-30T06:44:03.303ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
90212025-04-30T06:44:03.303ZWARNtest_unauthorized (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
90222025-04-30T06:44:03.305ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90232025-04-30T06:44:03.311ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
90242025-04-30T06:44:03.312ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
90252025-04-30T06:44:03.312ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
90262025-04-30T06:44:03.312ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
90272025-04-30T06:44:03.314ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90282025-04-30T06:44:03.321ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
90292025-04-30T06:44:03.321ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
90302025-04-30T06:44:03.321ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
90312025-04-30T06:44:03.322ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
90322025-04-30T06:44:03.324ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
90332025-04-30T06:44:03.324ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
90342025-04-30T06:44:03.325ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
90352025-04-30T06:44:03.385ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
90362025-04-30T06:44:03.385ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory
90372025-04-30T06:44:03.386ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:61417 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
90382025-04-30T06:44:03.386ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:61417 req_id = 27e1b011-db52-484e-8178-76a0af7e0738 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
90392025-04-30T06:44:03.388ZINFOtest_unauthorized (dropshot): request completed latency_us = 1184 local_addr = [::1]:57786 method = GET remote_addr = [::1]:61417 req_id = 27e1b011-db52-484e-8178-76a0af7e0738 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
90402025-04-30T06:44:03.388ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = http://[::1]:57786 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:57786/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "27e1b011-db52-484e-8178-76a0af7e0738", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
90412025-04-30T06:44:03.391ZDEBGtest_unauthorized (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
90422025-04-30T06:44:03.391ZDEBGtest_unauthorized (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
90432025-04-30T06:44:03.391ZDEBGtest_unauthorized (BackgroundTasks): finished collection background_task = inventory_collection
90442025-04-30T06:44:03.393ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
90452025-04-30T06:44:03.393ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
90462025-04-30T06:44:03.393ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
90472025-04-30T06:44:03.393ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
90482025-04-30T06:44:03.394ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
90492025-04-30T06:44:03.397ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90502025-04-30T06:44:03.403ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
90512025-04-30T06:44:03.403ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
90522025-04-30T06:44:03.403ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
90532025-04-30T06:44:03.404ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
90542025-04-30T06:44:03.433ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
90552025-04-30T06:44:03.447ZWARNtest_unauthorized (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
90562025-04-30T06:44:03.447ZWARNtest_unauthorized (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
90572025-04-30T06:44:03.447ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 490.514679ms iteration = 30 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
90582025-04-30T06:44:03.448ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 31 reason = Signaled
90592025-04-30T06:44:03.448ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
90602025-04-30T06:44:03.448ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
90612025-04-30T06:44:03.448ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
90622025-04-30T06:44:03.448ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
90632025-04-30T06:44:03.449ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
90642025-04-30T06:44:03.452ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90652025-04-30T06:44:03.457ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
90662025-04-30T06:44:03.457ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
90672025-04-30T06:44:03.457ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
90682025-04-30T06:44:03.458ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
90692025-04-30T06:44:03.474ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1a8b2318-abf8-406d-a389-112ab91a0de8, 0)]
90702025-04-30T06:44:03.474ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
90712025-04-30T06:44:03.475ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 72 23 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 r#..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:53826 req_id = 84b7fa2c-5fb9-4cde-8f1b-d2d0561ca03a
90722025-04-30T06:44:03.475ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 29219,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:53826 req_id = 84b7fa2c-5fb9-4cde-8f1b-d2d0561ca03a
90732025-04-30T06:44:03.475ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
90742025-04-30T06:44:03.475ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
90752025-04-30T06:44:03.476ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:53826 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 84b7fa2c-5fb9-4cde-8f1b-d2d0561ca03a
90762025-04-30T06:44:03.476ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4308, tv_nsec: 275512291 } })
90772025-04-30T06:44:03.476ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
90782025-04-30T06:44:03.477ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 36 9a 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e 6...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:49995 req_id = a7dd6386-e004-4b73-bcbd-23c394e234f2
90792025-04-30T06:44:03.477ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 13978,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:49995 req_id = a7dd6386-e004-4b73-bcbd-23c394e234f2
90802025-04-30T06:44:03.477ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
90812025-04-30T06:44:03.477ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:49995 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = a7dd6386-e004-4b73-bcbd-23c394e234f2
90822025-04-30T06:44:03.540ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
90832025-04-30T06:44:03.541ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
90842025-04-30T06:44:03.541ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
90852025-04-30T06:44:03.541ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
90862025-04-30T06:44:03.541ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
90872025-04-30T06:44:03.542ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
90882025-04-30T06:44:03.544ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 00 34 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c .4..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:65261 req_id = 87b515e7-939f-44b3-81b1-e030cf06f438
90892025-04-30T06:44:03.544ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 52,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:65261 req_id = 87b515e7-939f-44b3-81b1-e030cf06f438
90902025-04-30T06:44:03.544ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
90912025-04-30T06:44:03.544ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 52, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:65261 req_id = 87b515e7-939f-44b3-81b1-e030cf06f438
90922025-04-30T06:44:03.545ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
90932025-04-30T06:44:03.545ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
90942025-04-30T06:44:03.545ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
90952025-04-30T06:44:03.547ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90962025-04-30T06:44:03.555ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
90972025-04-30T06:44:03.555ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
90982025-04-30T06:44:03.555ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
90992025-04-30T06:44:03.555ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
91002025-04-30T06:44:03.563ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
91012025-04-30T06:44:03.563ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
91022025-04-30T06:44:03.564ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
91032025-04-30T06:44:03.574ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "31c1431c-c1f3-4ed5-b5d7-3d8ccaa66cc0", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
91042025-04-30T06:44:03.574ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
91052025-04-30T06:44:03.635ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "51e97384-7dfe-4c02-8b58-53c34e425afd", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
91062025-04-30T06:44:03.635ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
91072025-04-30T06:44:03.657ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "aeb7ccd8-98c3-4183-94f3-efba3978a4a3", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
91082025-04-30T06:44:03.657ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
91092025-04-30T06:44:03.661ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "57d6b6f7-1f3b-4757-b095-546d497e3fd2", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
91102025-04-30T06:44:03.662ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
91112025-04-30T06:44:03.666ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
91122025-04-30T06:44:03.668ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8154bba9-4b9c-4f73-8c05-448d0f2f99c0", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
91132025-04-30T06:44:03.669ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
91142025-04-30T06:44:03.674ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "72430920-4297-49b1-99ba-2b1e6e0fbc35", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
91152025-04-30T06:44:03.674ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
91162025-04-30T06:44:03.678ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1935c418-2118-4815-808a-7331e5908532", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
91172025-04-30T06:44:03.679ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
91182025-04-30T06:44:03.686ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d6fa1acb-a088-4438-92e7-40aef431fe2f", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
91192025-04-30T06:44:03.686ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
91202025-04-30T06:44:03.690ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "37fa173e-21ea-419d-967c-306cce209ec9", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
91212025-04-30T06:44:03.695ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
91222025-04-30T06:44:03.695ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0a01d590-03b6-46ca-b2da-8997c17d42a2", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
91232025-04-30T06:44:03.695ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
91242025-04-30T06:44:03.695ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "34f31f01-de14-4931-a18b-97af5b2fe4b0", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
91252025-04-30T06:44:03.695ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
91262025-04-30T06:44:03.697ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9df91ce3-2d12-41dd-b7b4-08bc3c0ecbbf", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
91272025-04-30T06:44:03.697ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
91282025-04-30T06:44:03.699ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "42853f1a-ba23-4bd6-b8a7-7412b8228c01", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
91292025-04-30T06:44:03.700ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
91302025-04-30T06:44:03.701ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0bd308b1-7666-42d6-84ae-4f8343499604", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
91312025-04-30T06:44:03.701ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
91322025-04-30T06:44:03.703ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "444f9b1c-56c1-41a1-9bf5-25701ca1e38b", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
91332025-04-30T06:44:03.703ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
91342025-04-30T06:44:03.705ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "744ff166-33f0-4566-aedc-772224448426", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
91352025-04-30T06:44:03.705ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
91362025-04-30T06:44:03.708ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c541b716-ec96-420a-9f49-c93546b3f390", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
91372025-04-30T06:44:03.708ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
91382025-04-30T06:44:03.710ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8f6aaaad-966f-4f60-a8d2-84c8859fc4d6", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
91392025-04-30T06:44:03.710ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
91402025-04-30T06:44:03.712ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3b171b1b-fd84-461d-8fe4-ea1205280b07", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
91412025-04-30T06:44:03.713ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
91422025-04-30T06:44:03.714ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d7fbd5be-2dac-4608-802a-5d3d7d4d2304", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
91432025-04-30T06:44:03.714ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
91442025-04-30T06:44:03.715ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8464fb17-431e-4281-b87b-246228c45ccd", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
91452025-04-30T06:44:03.715ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
91462025-04-30T06:44:03.716ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6d8162d4-3c62-483c-8cc5-6fbc1594b3de", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
91472025-04-30T06:44:03.716ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
91482025-04-30T06:44:03.717ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
91492025-04-30T06:44:03.717ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
91502025-04-30T06:44:03.717ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
91512025-04-30T06:44:03.717ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
91522025-04-30T06:44:03.718ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
91532025-04-30T06:44:03.718ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "95300500-b610-4268-a0c6-7e3728c2942b", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
91542025-04-30T06:44:03.718ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
91552025-04-30T06:44:03.727ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91562025-04-30T06:44:03.735ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
91572025-04-30T06:44:03.735ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
91582025-04-30T06:44:03.735ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
91592025-04-30T06:44:03.736ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
91602025-04-30T06:44:03.737ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "67d6b43d-6069-4644-8508-ee8d4e2d596a", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
91612025-04-30T06:44:03.738ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
91622025-04-30T06:44:03.738ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
91632025-04-30T06:44:03.738ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
91642025-04-30T06:44:03.739ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
91652025-04-30T06:44:03.743ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c510177b-758a-456f-bbed-11b788bce189", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
91662025-04-30T06:44:03.743ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
91672025-04-30T06:44:03.750ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f87bf380-cf0c-4dc7-95d4-331b8e990082", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
91682025-04-30T06:44:03.750ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
91692025-04-30T06:44:03.755ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "26fcb461-afb3-4683-8709-dc055445653c", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
91702025-04-30T06:44:03.755ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
91712025-04-30T06:44:03.758ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb881909-719d-48c5-a9be-4312ae8ed7e2", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
91722025-04-30T06:44:03.758ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
91732025-04-30T06:44:03.761ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ac4d927b-7913-46c4-a23c-2c5354a94152", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
91742025-04-30T06:44:03.762ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
91752025-04-30T06:44:03.764ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "195da1cf-d535-4dad-a0d0-f857e97116ec", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
91762025-04-30T06:44:03.765ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
91772025-04-30T06:44:03.767ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "54b8e4b1-d70b-4f26-9fd5-7c5e59e511d9", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
91782025-04-30T06:44:03.767ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
91792025-04-30T06:44:03.770ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "dcf37ac8-c89f-49be-8da9-b883396f38bd", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
91802025-04-30T06:44:03.771ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
91812025-04-30T06:44:03.774ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
91822025-04-30T06:44:03.775ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "dce47e76-be6c-4ba9-a66d-63eba4c4dad4", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
91832025-04-30T06:44:03.775ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
91842025-04-30T06:44:03.779ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "96814830-ed21-4088-8b1f-261d4c877051", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
91852025-04-30T06:44:03.779ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
91862025-04-30T06:44:03.780ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5db7b80f-d5f6-41bf-b8e6-fe9e2b64c00d", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
91872025-04-30T06:44:03.780ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
91882025-04-30T06:44:03.790ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "051bd882-3f38-485c-bfc3-a98b71d0c0c8", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
91892025-04-30T06:44:03.791ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
91902025-04-30T06:44:03.793ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f33ca800-9158-41f2-9e07-827ca4fd78dd", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
91912025-04-30T06:44:03.794ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
91922025-04-30T06:44:03.796ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9e478726-d252-48ea-8490-7547f677bd79", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
91932025-04-30T06:44:03.796ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
91942025-04-30T06:44:03.800ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3c63e921-0533-4125-a10f-f3478c4dda5f", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
91952025-04-30T06:44:03.800ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
91962025-04-30T06:44:03.803ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ec5afdbe-dbf1-48b6-8d9a-2ec08f45b946", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
91972025-04-30T06:44:03.803ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
91982025-04-30T06:44:03.805ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "57f25ff4-82fb-4243-a45e-ca5512fd173a", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
91992025-04-30T06:44:03.806ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
92002025-04-30T06:44:03.807ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1f0532f4-cd51-4274-b4c8-83f2aa7f1d50", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
92012025-04-30T06:44:03.807ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
92022025-04-30T06:44:03.808ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0a0f80d5-50cc-49c4-9c7e-975e96b21662", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
92032025-04-30T06:44:03.809ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
92042025-04-30T06:44:03.812ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4e01bf3d-19bb-4d30-a38e-1602f5b20f02", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
92052025-04-30T06:44:03.812ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
92062025-04-30T06:44:03.813ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1f6b8c90-d74d-40ba-aacf-c21e46d3fdb9", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
92072025-04-30T06:44:03.815ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
92082025-04-30T06:44:03.815ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
92092025-04-30T06:44:03.815ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
92102025-04-30T06:44:03.816ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
92112025-04-30T06:44:03.826ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
92122025-04-30T06:44:03.827ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
92132025-04-30T06:44:03.828ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
92142025-04-30T06:44:03.828ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
92152025-04-30T06:44:03.828ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
92162025-04-30T06:44:03.829ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
92172025-04-30T06:44:03.832ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92182025-04-30T06:44:03.839ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
92192025-04-30T06:44:03.839ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
92202025-04-30T06:44:03.839ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
92212025-04-30T06:44:03.839ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
92222025-04-30T06:44:03.839ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
92232025-04-30T06:44:03.840ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
92242025-04-30T06:44:03.842ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92252025-04-30T06:44:03.855ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
92262025-04-30T06:44:03.855ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
92272025-04-30T06:44:03.855ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
92282025-04-30T06:44:03.856ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
92292025-04-30T06:44:03.856ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92302025-04-30T06:44:03.860ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
92312025-04-30T06:44:03.860ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
92322025-04-30T06:44:03.860ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
92332025-04-30T06:44:03.861ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
92342025-04-30T06:44:03.863ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
92352025-04-30T06:44:03.863ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
92362025-04-30T06:44:03.863ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
92372025-04-30T06:44:03.933ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
92382025-04-30T06:44:03.933ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory
92392025-04-30T06:44:03.934ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:64295 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
92402025-04-30T06:44:03.936ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:64295 req_id = 37f31fa3-1f1d-4edd-ad9b-367340334205 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
92412025-04-30T06:44:03.938ZINFOtest_unauthorized (dropshot): request completed latency_us = 1190 local_addr = [::1]:57786 method = GET remote_addr = [::1]:64295 req_id = 37f31fa3-1f1d-4edd-ad9b-367340334205 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
92422025-04-30T06:44:03.938ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = http://[::1]:57786 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:57786/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "37f31fa3-1f1d-4edd-ad9b-367340334205", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
92432025-04-30T06:44:03.941ZDEBGtest_unauthorized (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
92442025-04-30T06:44:03.941ZDEBGtest_unauthorized (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
92452025-04-30T06:44:03.941ZDEBGtest_unauthorized (BackgroundTasks): finished collection background_task = inventory_collection
92462025-04-30T06:44:03.944ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
92472025-04-30T06:44:03.944ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
92482025-04-30T06:44:03.944ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
92492025-04-30T06:44:03.944ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
92502025-04-30T06:44:03.945ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
92512025-04-30T06:44:03.948ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92522025-04-30T06:44:03.951ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
92532025-04-30T06:44:03.952ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
92542025-04-30T06:44:03.952ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
92552025-04-30T06:44:03.952ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
92562025-04-30T06:44:03.991ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
92572025-04-30T06:44:04.009ZWARNtest_unauthorized (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
92582025-04-30T06:44:04.009ZWARNtest_unauthorized (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
92592025-04-30T06:44:04.009ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 561.769679ms iteration = 31 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
92602025-04-30T06:44:04.009ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 32 reason = Signaled
92612025-04-30T06:44:04.009ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
92622025-04-30T06:44:04.009ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
92632025-04-30T06:44:04.009ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
92642025-04-30T06:44:04.010ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
92652025-04-30T06:44:04.010ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
92662025-04-30T06:44:04.016ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92672025-04-30T06:44:04.022ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
92682025-04-30T06:44:04.022ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
92692025-04-30T06:44:04.022ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
92702025-04-30T06:44:04.022ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
92712025-04-30T06:44:04.041ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1a8b2318-abf8-406d-a389-112ab91a0de8, 0)]
92722025-04-30T06:44:04.041ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
92732025-04-30T06:44:04.042ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 9b 65 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .e..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:60835 req_id = 895c6931-a356-4e4a-8b15-3b6f26065e60
92742025-04-30T06:44:04.042ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 39781,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60835 req_id = 895c6931-a356-4e4a-8b15-3b6f26065e60
92752025-04-30T06:44:04.042ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
92762025-04-30T06:44:04.042ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
92772025-04-30T06:44:04.042ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:60835 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 895c6931-a356-4e4a-8b15-3b6f26065e60
92782025-04-30T06:44:04.042ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
92792025-04-30T06:44:04.042ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
92802025-04-30T06:44:04.042ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
92812025-04-30T06:44:04.043ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
92822025-04-30T06:44:04.044ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
92832025-04-30T06:44:04.044ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4308, tv_nsec: 843431092 } })
92842025-04-30T06:44:04.044ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
92852025-04-30T06:44:04.046ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 72 b8 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e r...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:52892 req_id = 22ea247a-02f4-4e7a-97d2-d3a19729cd05
92862025-04-30T06:44:04.046ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 29368,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52892 req_id = 22ea247a-02f4-4e7a-97d2-d3a19729cd05
92872025-04-30T06:44:04.047ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
92882025-04-30T06:44:04.047ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:52892 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 22ea247a-02f4-4e7a-97d2-d3a19729cd05
92892025-04-30T06:44:04.122ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
92902025-04-30T06:44:04.122ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92912025-04-30T06:44:04.127ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
92922025-04-30T06:44:04.127ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
92932025-04-30T06:44:04.127ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
92942025-04-30T06:44:04.128ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
92952025-04-30T06:44:04.131ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 80 ad 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:59813 req_id = b2b473a6-b68c-416f-851f-e1f6a9f0bb75
92962025-04-30T06:44:04.131ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 32941,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:59813 req_id = b2b473a6-b68c-416f-851f-e1f6a9f0bb75
92972025-04-30T06:44:04.131ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
92982025-04-30T06:44:04.132ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 32941, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:59813 req_id = b2b473a6-b68c-416f-851f-e1f6a9f0bb75
92992025-04-30T06:44:04.133ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
93002025-04-30T06:44:04.133ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
93012025-04-30T06:44:04.133ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
93022025-04-30T06:44:04.134ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
93032025-04-30T06:44:04.134ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
93042025-04-30T06:44:04.134ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
93052025-04-30T06:44:04.137ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "c24315db-0681-459b-89eb-6ba2fe144ce7", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
93062025-04-30T06:44:04.137ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
93072025-04-30T06:44:04.139ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "60afe7ff-8f56-4fb4-8117-c88366308194", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
93082025-04-30T06:44:04.140ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
93092025-04-30T06:44:04.143ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "69af677a-c143-4168-ad5b-91e2464c6793", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
93102025-04-30T06:44:04.143ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
93112025-04-30T06:44:04.146ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7dc9ca83-3470-4983-ba39-36c5a3625ca1", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
93122025-04-30T06:44:04.146ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
93132025-04-30T06:44:04.148ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c5e53611-ab8c-4d43-95e6-89479cb1c7c4", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
93142025-04-30T06:44:04.148ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
93152025-04-30T06:44:04.150ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2b55bdf9-b00e-49f7-91ee-a86d068ee949", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
93162025-04-30T06:44:04.150ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
93172025-04-30T06:44:04.152ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b6d7a569-af79-4b56-8283-90f71b88b5aa", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
93182025-04-30T06:44:04.152ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
93192025-04-30T06:44:04.154ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "89b36d50-721c-408d-b53f-2ed1318f5bea", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
93202025-04-30T06:44:04.155ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
93212025-04-30T06:44:04.156ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2a0572ee-1e62-4270-8c76-8ee4e80a5155", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
93222025-04-30T06:44:04.156ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
93232025-04-30T06:44:04.158ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fd05be31-b35f-44dd-9039-31c944718635", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
93242025-04-30T06:44:04.158ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
93252025-04-30T06:44:04.162ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5a8c3ef3-508b-4d6e-ba08-c7c199a74c2f", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
93262025-04-30T06:44:04.162ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
93272025-04-30T06:44:04.163ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
93282025-04-30T06:44:04.163ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5ff17bfd-6b94-4a27-bf2a-59acb8e8a920", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
93292025-04-30T06:44:04.164ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
93302025-04-30T06:44:04.165ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e9a54693-4ae7-411d-bb7f-c844b5bd944d", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
93312025-04-30T06:44:04.165ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
93322025-04-30T06:44:04.167ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "aa0fbb37-446f-42cc-81ef-5e269653b06d", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
93332025-04-30T06:44:04.168ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
93342025-04-30T06:44:04.169ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ce5cf387-d4e9-4001-88fb-df7b1274e911", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
93352025-04-30T06:44:04.169ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
93362025-04-30T06:44:04.172ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0d7f8fbb-211b-4c3e-a6fe-a3e014def395", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
93372025-04-30T06:44:04.172ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
93382025-04-30T06:44:04.173ZDEBGtest_unauthorized (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
93392025-04-30T06:44:04.173ZDEBGtest_unauthorized (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
93402025-04-30T06:44:04.173ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a611687a-b1c3-49af-b16d-f01ccd5c56ff", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
93412025-04-30T06:44:04.173ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
93422025-04-30T06:44:04.173ZDEBGtest_unauthorized (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a
93432025-04-30T06:44:04.174ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
93442025-04-30T06:44:04.179ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 2 table_name = fields_string
93452025-04-30T06:44:04.179ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
93462025-04-30T06:44:04.182ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 1 table_name = measurements_i64
93472025-04-30T06:44:04.182ZTRACtest_unauthorized (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
93482025-04-30T06:44:04.183ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a4b722f0-7a2e-4ce8-b9c5-cc31ce162b56", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
93492025-04-30T06:44:04.183ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
93502025-04-30T06:44:04.186ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "549ccdfb-332a-47d8-9b73-c1b9e41b8d41", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
93512025-04-30T06:44:04.187ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
93522025-04-30T06:44:04.188ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c2a7c5ee-453e-4346-b729-479cca5aa134", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
93532025-04-30T06:44:04.188ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
93542025-04-30T06:44:04.189ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "25af5fd5-761b-4dea-84fc-30e8d7c744b3", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
93552025-04-30T06:44:04.190ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
93562025-04-30T06:44:04.192ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "137968c7-adfc-46b0-b1c3-28a93ff6e99c", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
93572025-04-30T06:44:04.192ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
93582025-04-30T06:44:04.193ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "06efc02d-7ff5-4296-af98-78e52018edb0", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
93592025-04-30T06:44:04.194ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
93602025-04-30T06:44:04.195ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5755d5c1-056e-46f4-b8dc-8d8615b4d653", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
93612025-04-30T06:44:04.195ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
93622025-04-30T06:44:04.196ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b93fb46c-a414-425b-bc32-8e412c46d2e2", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
93632025-04-30T06:44:04.197ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
93642025-04-30T06:44:04.198ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa084dc3-a297-4883-acbc-906f079a99fd", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
93652025-04-30T06:44:04.199ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
93662025-04-30T06:44:04.200ZDEBGtest_unauthorized (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
93672025-04-30T06:44:04.200ZDEBGtest_unauthorized (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
93682025-04-30T06:44:04.202ZTRACtest_unauthorized (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
93692025-04-30T06:44:04.203ZDEBGtest_unauthorized (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
93702025-04-30T06:44:04.203ZTRACtest_unauthorized (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
93712025-04-30T06:44:04.203ZTRACtest_unauthorized (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
93722025-04-30T06:44:04.204ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "03d7ac4c-282b-472a-8936-ea6f32c33902", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
93732025-04-30T06:44:04.206ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
93742025-04-30T06:44:04.210ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d199e001-1a4e-43e7-b9ad-c6dfbb0b7ec5", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
93752025-04-30T06:44:04.211ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
93762025-04-30T06:44:04.212ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a7b938e0-23a5-4f0c-95a3-ef5743812e7d", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
93772025-04-30T06:44:04.213ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
93782025-04-30T06:44:04.214ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
93792025-04-30T06:44:04.214ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
93802025-04-30T06:44:04.214ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
93812025-04-30T06:44:04.214ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
93822025-04-30T06:44:04.215ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
93832025-04-30T06:44:04.216ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "df98d2e0-480b-4c92-95e4-7513aafdfdd7", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
93842025-04-30T06:44:04.216ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
93852025-04-30T06:44:04.217ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e8ef47d8-8952-491e-96e4-be450dbf8c06", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
93862025-04-30T06:44:04.217ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
93872025-04-30T06:44:04.219ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93882025-04-30T06:44:04.226ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
93892025-04-30T06:44:04.226ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
93902025-04-30T06:44:04.226ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
93912025-04-30T06:44:04.226ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
93922025-04-30T06:44:04.227ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "70f461a1-d782-4dde-9b0d-7f274b19711e", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
93932025-04-30T06:44:04.228ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
93942025-04-30T06:44:04.229ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "90542210-ebf0-4ad9-b69a-94d4a8012272", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
93952025-04-30T06:44:04.229ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
93962025-04-30T06:44:04.229ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
93972025-04-30T06:44:04.230ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
93982025-04-30T06:44:04.230ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
93992025-04-30T06:44:04.232ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f2c39237-215c-4daf-913c-4830dc5af698", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
94002025-04-30T06:44:04.232ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
94012025-04-30T06:44:04.234ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5630960b-438a-4945-9d9d-39d4f0b825d2", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
94022025-04-30T06:44:04.234ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
94032025-04-30T06:44:04.236ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2ef506a4-c9c0-4900-8348-baa87ecdde27", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
94042025-04-30T06:44:04.236ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
94052025-04-30T06:44:04.241ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6e21a389-b85e-49d1-8ee6-8df702e9659b", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
94062025-04-30T06:44:04.241ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
94072025-04-30T06:44:04.243ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4a5f6104-b2a5-4b32-bf28-06d11f0d9f1f", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
94082025-04-30T06:44:04.243ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
94092025-04-30T06:44:04.245ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "911d23bd-f58a-4636-85a0-a7fcf861b6fa", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
94102025-04-30T06:44:04.245ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
94112025-04-30T06:44:04.248ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "62c9303b-e3bc-430f-b11e-3216bc65b2c3", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
94122025-04-30T06:44:04.248ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
94132025-04-30T06:44:04.251ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9f2921f5-5e64-4bba-9c41-c458ff6cf3da", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
94142025-04-30T06:44:04.251ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
94152025-04-30T06:44:04.254ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8c45809a-273d-4ee6-b76a-eb2a3296eaa9", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
94162025-04-30T06:44:04.254ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
94172025-04-30T06:44:04.255ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "aab380ec-b4ad-4838-bffc-5871f2c628ff", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
94182025-04-30T06:44:04.256ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
94192025-04-30T06:44:04.259ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ddb819ad-a412-4407-a150-345592021e17", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
94202025-04-30T06:44:04.259ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
94212025-04-30T06:44:04.261ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8e84a7e8-2fcb-4710-9dca-2df84cf474a4", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
94222025-04-30T06:44:04.261ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
94232025-04-30T06:44:04.261ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
94242025-04-30T06:44:04.261ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
94252025-04-30T06:44:04.261ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
94262025-04-30T06:44:04.262ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
94272025-04-30T06:44:04.265ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94282025-04-30T06:44:04.270ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
94292025-04-30T06:44:04.270ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
94302025-04-30T06:44:04.270ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
94312025-04-30T06:44:04.270ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
94322025-04-30T06:44:04.270ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
94332025-04-30T06:44:04.271ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
94342025-04-30T06:44:04.274ZWARNtest_unauthorized (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
94352025-04-30T06:44:04.276ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
94362025-04-30T06:44:04.276ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94372025-04-30T06:44:04.283ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
94382025-04-30T06:44:04.283ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
94392025-04-30T06:44:04.283ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
94402025-04-30T06:44:04.283ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
94412025-04-30T06:44:04.360ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
94422025-04-30T06:44:04.360ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory
94432025-04-30T06:44:04.362ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
94442025-04-30T06:44:04.362ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
94452025-04-30T06:44:04.362ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
94462025-04-30T06:44:04.362ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
94472025-04-30T06:44:04.363ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
94482025-04-30T06:44:04.363ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:60077 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
94492025-04-30T06:44:04.366ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:60077 req_id = 4f0f3dc6-8c79-453a-b896-2a49820e29fc sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
94502025-04-30T06:44:04.366ZINFOtest_unauthorized (dropshot): request completed latency_us = 642 local_addr = [::1]:57786 method = GET remote_addr = [::1]:60077 req_id = 4f0f3dc6-8c79-453a-b896-2a49820e29fc response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
94512025-04-30T06:44:04.367ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = http://[::1]:57786 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:57786/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "4f0f3dc6-8c79-453a-b896-2a49820e29fc", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
94522025-04-30T06:44:04.371ZDEBGtest_unauthorized (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
94532025-04-30T06:44:04.371ZDEBGtest_unauthorized (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
94542025-04-30T06:44:04.371ZDEBGtest_unauthorized (BackgroundTasks): finished collection background_task = inventory_collection
94552025-04-30T06:44:04.375ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
94562025-04-30T06:44:04.376ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
94572025-04-30T06:44:04.376ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
94582025-04-30T06:44:04.376ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
94592025-04-30T06:44:04.377ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
94602025-04-30T06:44:04.377ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94612025-04-30T06:44:04.395ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
94622025-04-30T06:44:04.395ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
94632025-04-30T06:44:04.395ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
94642025-04-30T06:44:04.396ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
94652025-04-30T06:44:04.398ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94662025-04-30T06:44:04.406ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
94672025-04-30T06:44:04.408ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
94682025-04-30T06:44:04.408ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
94692025-04-30T06:44:04.410ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
94702025-04-30T06:44:04.410ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
94712025-04-30T06:44:04.410ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
94722025-04-30T06:44:04.419ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
94732025-04-30T06:44:04.480ZWARNtest_unauthorized (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
94742025-04-30T06:44:04.481ZWARNtest_unauthorized (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
94752025-04-30T06:44:04.481ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 471.333331ms iteration = 32 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
94762025-04-30T06:44:04.481ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 33 reason = Signaled
94772025-04-30T06:44:04.481ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
94782025-04-30T06:44:04.481ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
94792025-04-30T06:44:04.481ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
94802025-04-30T06:44:04.481ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
94812025-04-30T06:44:04.485ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
94822025-04-30T06:44:04.490ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
94832025-04-30T06:44:04.490ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94842025-04-30T06:44:04.497ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
94852025-04-30T06:44:04.497ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
94862025-04-30T06:44:04.497ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
94872025-04-30T06:44:04.498ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
94882025-04-30T06:44:04.510ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1a8b2318-abf8-406d-a389-112ab91a0de8, 0)]
94892025-04-30T06:44:04.510ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
94902025-04-30T06:44:04.511ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 76 40 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 v@..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:63300 req_id = 991a9ca3-c542-4a17-bcd8-48e6d411aa3b
94912025-04-30T06:44:04.511ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 30272,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63300 req_id = 991a9ca3-c542-4a17-bcd8-48e6d411aa3b
94922025-04-30T06:44:04.511ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
94932025-04-30T06:44:04.511ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
94942025-04-30T06:44:04.511ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:63300 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 991a9ca3-c542-4a17-bcd8-48e6d411aa3b
94952025-04-30T06:44:04.512ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4309, tv_nsec: 311247165 } })
94962025-04-30T06:44:04.512ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
94972025-04-30T06:44:04.512ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 6d 6a 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e mj..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:53247 req_id = d2c4c9da-f18d-4055-8a05-e17ff94626e5
94982025-04-30T06:44:04.512ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 28010,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:53247 req_id = d2c4c9da-f18d-4055-8a05-e17ff94626e5
94992025-04-30T06:44:04.512ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
95002025-04-30T06:44:04.512ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:53247 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = d2c4c9da-f18d-4055-8a05-e17ff94626e5
95012025-04-30T06:44:04.568ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
95022025-04-30T06:44:04.569ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
95032025-04-30T06:44:04.569ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
95042025-04-30T06:44:04.569ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
95052025-04-30T06:44:04.569ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
95062025-04-30T06:44:04.570ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
95072025-04-30T06:44:04.571ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: eb 9f 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:51297 req_id = a97d4466-16ab-4b3f-ab33-5d23f8705591
95082025-04-30T06:44:04.571ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 60319,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51297 req_id = a97d4466-16ab-4b3f-ab33-5d23f8705591
95092025-04-30T06:44:04.571ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
95102025-04-30T06:44:04.571ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 60319, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:51297 req_id = a97d4466-16ab-4b3f-ab33-5d23f8705591
95112025-04-30T06:44:04.572ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
95122025-04-30T06:44:04.572ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
95132025-04-30T06:44:04.572ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
95142025-04-30T06:44:04.573ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95152025-04-30T06:44:04.579ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
95162025-04-30T06:44:04.579ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
95172025-04-30T06:44:04.579ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
95182025-04-30T06:44:04.580ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
95192025-04-30T06:44:04.597ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
95202025-04-30T06:44:04.597ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
95212025-04-30T06:44:04.598ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
95222025-04-30T06:44:04.608ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f58207b-4c3f-410d-9ab7-ea51ff4c8cd4", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
95232025-04-30T06:44:04.609ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
95242025-04-30T06:44:04.643ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "dcd9e30a-9bc4-4b95-8179-1b8ba71e0376", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
95252025-04-30T06:44:04.643ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
95262025-04-30T06:44:04.659ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "06054c84-80fe-4093-9e94-8304bed2e6b3", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
95272025-04-30T06:44:04.659ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
95282025-04-30T06:44:04.664ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "dec13afd-f975-436a-aa5b-e52e3b08cf50", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
95292025-04-30T06:44:04.664ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
95302025-04-30T06:44:04.669ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e77505ca-bacc-4671-a12a-aa55d34d90cc", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
95312025-04-30T06:44:04.669ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
95322025-04-30T06:44:04.672ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
95332025-04-30T06:44:04.673ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "05d01e4b-33f7-443f-b30e-360bc26dcd19", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
95342025-04-30T06:44:04.673ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
95352025-04-30T06:44:04.676ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fd56cca1-aefe-46be-82f3-da70751c5872", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
95362025-04-30T06:44:04.676ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
95372025-04-30T06:44:04.677ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "73e2467f-a339-42e6-ab68-6154fa76c0a7", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
95382025-04-30T06:44:04.678ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
95392025-04-30T06:44:04.679ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d1d44f40-c66f-40f0-ba2a-80da0c65f6f7", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
95402025-04-30T06:44:04.679ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
95412025-04-30T06:44:04.680ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "dd19460b-7a8a-41e9-a8ba-a8b6cf138bac", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
95422025-04-30T06:44:04.680ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
95432025-04-30T06:44:04.681ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "384e4a80-88da-4b81-bd6f-e52a3b221d7b", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
95442025-04-30T06:44:04.681ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
95452025-04-30T06:44:04.682ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd664267-5332-4320-8fa5-4d35cc28a9c9", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
95462025-04-30T06:44:04.682ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
95472025-04-30T06:44:04.683ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3a4a5711-d76e-42eb-b671-d401451a9863", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
95482025-04-30T06:44:04.684ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
95492025-04-30T06:44:04.686ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8b4482e0-4feb-45a3-8fed-31e675259a5b", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
95502025-04-30T06:44:04.686ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
95512025-04-30T06:44:04.690ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "54921fa7-6bb3-49cf-939f-c44078405a51", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
95522025-04-30T06:44:04.690ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
95532025-04-30T06:44:04.694ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "18953b9b-aa8d-461a-ac21-f1d5671659ee", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
95542025-04-30T06:44:04.694ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
95552025-04-30T06:44:04.697ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fd388d9e-027e-424a-8b5f-1e29add24105", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
95562025-04-30T06:44:04.697ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
95572025-04-30T06:44:04.700ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "542520ea-2ca4-4d3a-a088-936d01cbd86d", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
95582025-04-30T06:44:04.700ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
95592025-04-30T06:44:04.702ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7e307bd1-db24-4046-befe-e69622e28b7c", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
95602025-04-30T06:44:04.702ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
95612025-04-30T06:44:04.703ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "35639372-62f0-4958-8e54-3044d3108dd9", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
95622025-04-30T06:44:04.703ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
95632025-04-30T06:44:04.704ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8c69ab72-9925-459a-91aa-56cb1987bb2d", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
95642025-04-30T06:44:04.704ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
95652025-04-30T06:44:04.706ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc452072-1f4b-4c6c-9285-c13e32482a75", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
95662025-04-30T06:44:04.706ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
95672025-04-30T06:44:04.707ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f0beb64-15ba-4417-992b-d940925de771", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
95682025-04-30T06:44:04.707ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
95692025-04-30T06:44:04.709ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1965b089-b80c-44d2-83d5-65a2c8cef47e", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
95702025-04-30T06:44:04.709ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
95712025-04-30T06:44:04.711ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "923c3043-d5df-432e-822a-f50116cb2da2", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
95722025-04-30T06:44:04.711ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
95732025-04-30T06:44:04.714ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b35aa880-ca5e-4071-bbc0-a4982bd0c3b1", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
95742025-04-30T06:44:04.714ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
95752025-04-30T06:44:04.717ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c2d42af9-50d0-4d07-9c7c-5547ee55412a", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
95762025-04-30T06:44:04.718ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
95772025-04-30T06:44:04.721ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7f846b48-4d18-436e-827e-7e8c84c376b6", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
95782025-04-30T06:44:04.721ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
95792025-04-30T06:44:04.723ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
95802025-04-30T06:44:04.723ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
95812025-04-30T06:44:04.723ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
95822025-04-30T06:44:04.723ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
95832025-04-30T06:44:04.723ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
95842025-04-30T06:44:04.725ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "47c401b9-1782-4df9-a7a2-29c45f3b68ba", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
95852025-04-30T06:44:04.725ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
95862025-04-30T06:44:04.726ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95872025-04-30T06:44:04.731ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
95882025-04-30T06:44:04.731ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
95892025-04-30T06:44:04.731ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
95902025-04-30T06:44:04.732ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
95912025-04-30T06:44:04.732ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0fef6358-2b4d-462b-9a79-8989965feccc", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
95922025-04-30T06:44:04.732ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
95932025-04-30T06:44:04.734ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "74e3decf-1d88-49e7-a131-533f06e0cede", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
95942025-04-30T06:44:04.734ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
95952025-04-30T06:44:04.734ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
95962025-04-30T06:44:04.734ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
95972025-04-30T06:44:04.735ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
95982025-04-30T06:44:04.736ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "21efc607-5c08-46be-9408-ea3424129718", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
95992025-04-30T06:44:04.736ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
96002025-04-30T06:44:04.740ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "806cd3b3-d2a6-4236-8871-8cf0742fc29d", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
96012025-04-30T06:44:04.740ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
96022025-04-30T06:44:04.741ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4a9d5b88-9d0e-4a6d-b29c-81bb9f8c8064", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
96032025-04-30T06:44:04.741ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
96042025-04-30T06:44:04.745ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c9937255-403d-4ab3-8641-f31ed5253735", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
96052025-04-30T06:44:04.745ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
96062025-04-30T06:44:04.747ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c472ff91-6682-4cbb-a216-016b44f476db", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
96072025-04-30T06:44:04.748ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
96082025-04-30T06:44:04.750ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d2f2a486-e770-4fdb-bb18-5c610e675ef9", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
96092025-04-30T06:44:04.750ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
96102025-04-30T06:44:04.753ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8654bfdb-7fe7-47de-9072-27026724f04f", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
96112025-04-30T06:44:04.753ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
96122025-04-30T06:44:04.755ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "072af837-00d8-445e-ae50-ce47ea85485f", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
96132025-04-30T06:44:04.755ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
96142025-04-30T06:44:04.765ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7817de4f-4337-4154-86d2-416da3cf0adf", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
96152025-04-30T06:44:04.765ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
96162025-04-30T06:44:04.769ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e64b3cd5-f6a4-4761-aca5-c6a153f21ed9", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
96172025-04-30T06:44:04.769ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
96182025-04-30T06:44:04.771ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
96192025-04-30T06:44:04.775ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "29ad8a07-0688-4f43-8442-143a726746bd", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
96202025-04-30T06:44:04.775ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
96212025-04-30T06:44:04.778ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b7a3bf45-181e-46b9-9230-4197c67b58d4", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
96222025-04-30T06:44:04.778ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
96232025-04-30T06:44:04.780ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb77e911-8a76-4790-9a9f-9a2b81d5d0dd", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
96242025-04-30T06:44:04.780ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
96252025-04-30T06:44:04.782ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e9310071-abd1-41b1-8222-939e58c9ec72", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
96262025-04-30T06:44:04.782ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
96272025-04-30T06:44:04.783ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
96282025-04-30T06:44:04.783ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
96292025-04-30T06:44:04.783ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
96302025-04-30T06:44:04.784ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
96312025-04-30T06:44:04.789ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96322025-04-30T06:44:04.794ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
96332025-04-30T06:44:04.794ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
96342025-04-30T06:44:04.794ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
96352025-04-30T06:44:04.795ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
96362025-04-30T06:44:04.795ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
96372025-04-30T06:44:04.795ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
96382025-04-30T06:44:04.799ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96392025-04-30T06:44:04.803ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
96402025-04-30T06:44:04.803ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
96412025-04-30T06:44:04.803ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
96422025-04-30T06:44:04.805ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
96432025-04-30T06:44:04.863ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
96442025-04-30T06:44:04.863ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory
96452025-04-30T06:44:04.864ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
96462025-04-30T06:44:04.864ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
96472025-04-30T06:44:04.864ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
96482025-04-30T06:44:04.864ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
96492025-04-30T06:44:04.905ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
96502025-04-30T06:44:04.905ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:51725 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
96512025-04-30T06:44:04.907ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:51725 req_id = 4db8e2f7-c0a2-48cb-a221-7cc2526cab19 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
96522025-04-30T06:44:04.907ZINFOtest_unauthorized (dropshot): request completed latency_us = 681 local_addr = [::1]:57786 method = GET remote_addr = [::1]:51725 req_id = 4db8e2f7-c0a2-48cb-a221-7cc2526cab19 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
96532025-04-30T06:44:04.908ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = http://[::1]:57786 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:57786/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "4db8e2f7-c0a2-48cb-a221-7cc2526cab19", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
96542025-04-30T06:44:04.913ZDEBGtest_unauthorized (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
96552025-04-30T06:44:04.913ZDEBGtest_unauthorized (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
96562025-04-30T06:44:04.913ZDEBGtest_unauthorized (BackgroundTasks): finished collection background_task = inventory_collection
96572025-04-30T06:44:04.915ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
96582025-04-30T06:44:04.915ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
96592025-04-30T06:44:04.915ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
96602025-04-30T06:44:04.915ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
96612025-04-30T06:44:04.916ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
96622025-04-30T06:44:04.917ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96632025-04-30T06:44:04.923ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
96642025-04-30T06:44:04.923ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
96652025-04-30T06:44:04.923ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
96662025-04-30T06:44:04.924ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
96672025-04-30T06:44:04.930ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96682025-04-30T06:44:04.937ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
96692025-04-30T06:44:04.938ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
96702025-04-30T06:44:04.938ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
96712025-04-30T06:44:04.938ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
96722025-04-30T06:44:04.940ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
96732025-04-30T06:44:04.940ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
96742025-04-30T06:44:04.942ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
96752025-04-30T06:44:04.989ZWARNtest_unauthorized (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
96762025-04-30T06:44:04.990ZWARNtest_unauthorized (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
96772025-04-30T06:44:04.990ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 509.132993ms iteration = 33 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
96782025-04-30T06:44:04.990ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 34 reason = Signaled
96792025-04-30T06:44:04.990ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
96802025-04-30T06:44:04.990ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
96812025-04-30T06:44:04.990ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
96822025-04-30T06:44:04.990ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
96832025-04-30T06:44:04.991ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
96842025-04-30T06:44:04.995ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96852025-04-30T06:44:05.001ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
96862025-04-30T06:44:05.001ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
96872025-04-30T06:44:05.001ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
96882025-04-30T06:44:05.002ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
96892025-04-30T06:44:05.007ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
96902025-04-30T06:44:05.007ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1a8b2318-abf8-406d-a389-112ab91a0de8, 0)]
96912025-04-30T06:44:05.007ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
96922025-04-30T06:44:05.009ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: b7 34 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .4..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:63111 req_id = 9685f531-2cba-4457-baaf-9c1429bd2f41
96932025-04-30T06:44:05.009ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 46900,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63111 req_id = 9685f531-2cba-4457-baaf-9c1429bd2f41
96942025-04-30T06:44:05.009ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
96952025-04-30T06:44:05.009ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
96962025-04-30T06:44:05.009ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:63111 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 9685f531-2cba-4457-baaf-9c1429bd2f41
96972025-04-30T06:44:05.010ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4309, tv_nsec: 809185163 } })
96982025-04-30T06:44:05.010ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
96992025-04-30T06:44:05.010ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: de ba 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:51023 req_id = b7a2b7ee-0ff7-4b4a-b59d-1bb9be528a29
97002025-04-30T06:44:05.010ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 57018,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51023 req_id = b7a2b7ee-0ff7-4b4a-b59d-1bb9be528a29
97012025-04-30T06:44:05.010ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
97022025-04-30T06:44:05.011ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:51023 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = b7a2b7ee-0ff7-4b4a-b59d-1bb9be528a29
97032025-04-30T06:44:05.062ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
97042025-04-30T06:44:05.063ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
97052025-04-30T06:44:05.063ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
97062025-04-30T06:44:05.063ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
97072025-04-30T06:44:05.063ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
97082025-04-30T06:44:05.064ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
97092025-04-30T06:44:05.067ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 69 16 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c i............_cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:61526 req_id = bccf9e3f-9c4f-4717-9756-f25a3e4452ae
97102025-04-30T06:44:05.067ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 26902,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:61526 req_id = bccf9e3f-9c4f-4717-9756-f25a3e4452ae
97112025-04-30T06:44:05.067ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
97122025-04-30T06:44:05.067ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 26902, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:61526 req_id = bccf9e3f-9c4f-4717-9756-f25a3e4452ae
97132025-04-30T06:44:05.067ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
97142025-04-30T06:44:05.067ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
97152025-04-30T06:44:05.068ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
97162025-04-30T06:44:05.097ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97172025-04-30T06:44:05.103ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
97182025-04-30T06:44:05.103ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
97192025-04-30T06:44:05.103ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
97202025-04-30T06:44:05.103ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
97212025-04-30T06:44:05.107ZINFOtest_unauthorized (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
97222025-04-30T06:44:05.108ZDEBGtest_unauthorized (oximeter): client request body = None method = GET uri = http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
97232025-04-30T06:44:05.108ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
97242025-04-30T06:44:05.108ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
97252025-04-30T06:44:05.109ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
97262025-04-30T06:44:05.110ZTRACtest_unauthorized (dropshot_internal): incoming request local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 04ed5744-f039-41b6-a9d3-328317da792d uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
97272025-04-30T06:44:05.111ZTRACtest_unauthorized (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 04ed5744-f039-41b6-a9d3-328317da792d resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
97282025-04-30T06:44:05.111ZDEBGtest_unauthorized (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 04ed5744-f039-41b6-a9d3-328317da792d roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
97292025-04-30T06:44:05.111ZDEBGtest_unauthorized (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 04ed5744-f039-41b6-a9d3-328317da792d resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
97302025-04-30T06:44:05.112ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "5af84317-e4e9-4bb6-8ba4-fc6b072ad40c", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
97312025-04-30T06:44:05.112ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
97322025-04-30T06:44:05.117ZINFOtest_unauthorized (dropshot_internal): request completed latency_us = 7825 local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 04ed5744-f039-41b6-a9d3-328317da792d response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
97332025-04-30T06:44:05.118ZDEBGtest_unauthorized (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "04ed5744-f039-41b6-a9d3-328317da792d", "content-length": "405", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
97342025-04-30T06:44:05.118ZDEBGtest_unauthorized (oximeter): client request body = None method = GET uri = http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
97352025-04-30T06:44:05.119ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ba19824a-8588-426d-aa2d-b6b2582d5d65", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
97362025-04-30T06:44:05.119ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
97372025-04-30T06:44:05.120ZTRACtest_unauthorized (dropshot_internal): incoming request local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = d8b35449-803a-41b5-ade0-22c11effa4cb uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
97382025-04-30T06:44:05.122ZTRACtest_unauthorized (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = d8b35449-803a-41b5-ade0-22c11effa4cb resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
97392025-04-30T06:44:05.122ZDEBGtest_unauthorized (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = d8b35449-803a-41b5-ade0-22c11effa4cb roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
97402025-04-30T06:44:05.123ZDEBGtest_unauthorized (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = d8b35449-803a-41b5-ade0-22c11effa4cb resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
97412025-04-30T06:44:05.126ZINFOtest_unauthorized (dropshot_internal): request completed latency_us = 6452 local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = d8b35449-803a-41b5-ade0-22c11effa4cb response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
97422025-04-30T06:44:05.127ZDEBGtest_unauthorized (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "d8b35449-803a-41b5-ade0-22c11effa4cb", "content-length": "29", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
97432025-04-30T06:44:05.127ZTRACtest_unauthorized (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 8d752adb-08b9-405f-a11b-66019bf447b5
97442025-04-30T06:44:05.127ZTRACtest_unauthorized (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
97452025-04-30T06:44:05.127ZINFOtest_unauthorized (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
97462025-04-30T06:44:05.131ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a59ed233-f42e-42bf-a804-3079a66d8171", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
97472025-04-30T06:44:05.131ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
97482025-04-30T06:44:05.146ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4251e5d5-18cb-4090-af9a-db5c3bf9c6f5", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
97492025-04-30T06:44:05.147ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
97502025-04-30T06:44:05.152ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6aeaef84-e447-4e91-8c4c-b70ca22edae1", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
97512025-04-30T06:44:05.154ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
97522025-04-30T06:44:05.157ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
97532025-04-30T06:44:05.159ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "dcfb7bf1-bb20-4e45-ab9b-aa6aa27091ea", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
97542025-04-30T06:44:05.160ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
97552025-04-30T06:44:05.163ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4a59cbba-44a9-4db2-bd40-a71873aff6f2", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
97562025-04-30T06:44:05.164ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
97572025-04-30T06:44:05.171ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9baff5f1-d093-4d92-b9fc-0bd06702ecb4", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
97582025-04-30T06:44:05.171ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
97592025-04-30T06:44:05.172ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ee69c8a5-4c19-4845-a8d9-44502866fb21", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
97602025-04-30T06:44:05.174ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
97612025-04-30T06:44:05.174ZDEBGtest_unauthorized (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
97622025-04-30T06:44:05.174ZDEBGtest_unauthorized (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
97632025-04-30T06:44:05.175ZDEBGtest_unauthorized (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a
97642025-04-30T06:44:05.179ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
97652025-04-30T06:44:05.180ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d452516c-ea62-4fcf-824a-a98e1dbaad50", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
97662025-04-30T06:44:05.180ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
97672025-04-30T06:44:05.182ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "03a06626-14be-47cd-abe7-d8629f583238", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
97682025-04-30T06:44:05.182ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
97692025-04-30T06:44:05.183ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 2 table_name = fields_string
97702025-04-30T06:44:05.183ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
97712025-04-30T06:44:05.184ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0589334d-8718-4026-b89f-0dff3b822edc", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
97722025-04-30T06:44:05.184ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
97732025-04-30T06:44:05.187ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "79cb6785-bd04-4c89-af20-9a10f19b14a8", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
97742025-04-30T06:44:05.188ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
97752025-04-30T06:44:05.191ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 1 table_name = measurements_i64
97762025-04-30T06:44:05.191ZTRACtest_unauthorized (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
97772025-04-30T06:44:05.194ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2e236882-30cf-4771-9731-6d7a903122bf", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
97782025-04-30T06:44:05.194ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
97792025-04-30T06:44:05.219ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
97802025-04-30T06:44:05.220ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
97812025-04-30T06:44:05.220ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
97822025-04-30T06:44:05.220ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
97832025-04-30T06:44:05.221ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
97842025-04-30T06:44:05.221ZDEBGtest_unauthorized (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
97852025-04-30T06:44:05.221ZDEBGtest_unauthorized (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
97862025-04-30T06:44:05.263ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97872025-04-30T06:44:05.269ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
97882025-04-30T06:44:05.269ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
97892025-04-30T06:44:05.269ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
97902025-04-30T06:44:05.269ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
97912025-04-30T06:44:05.270ZTRACtest_unauthorized (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
97922025-04-30T06:44:05.271ZDEBGtest_unauthorized (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
97932025-04-30T06:44:05.271ZTRACtest_unauthorized (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
97942025-04-30T06:44:05.271ZTRACtest_unauthorized (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
97952025-04-30T06:44:05.271ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f3a4e0e7-0b0a-4e12-b477-c0b20482d09f", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
97962025-04-30T06:44:05.271ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
97972025-04-30T06:44:05.273ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
97982025-04-30T06:44:05.273ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
97992025-04-30T06:44:05.274ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
98002025-04-30T06:44:05.275ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6cfa455f-9ecf-4948-8303-ae3e482c060f", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
98012025-04-30T06:44:05.275ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
98022025-04-30T06:44:05.276ZWARNtest_unauthorized (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
98032025-04-30T06:44:05.279ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "caca45eb-de97-4350-84c4-8dd25dfbeb26", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
98042025-04-30T06:44:05.280ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
98052025-04-30T06:44:05.284ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b2578b52-27e9-4a4e-a8a5-c96e297b0425", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
98062025-04-30T06:44:05.285ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
98072025-04-30T06:44:05.299ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "68624573-b010-4361-848b-55e0c1550aff", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
98082025-04-30T06:44:05.299ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
98092025-04-30T06:44:05.301ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fb0e36f9-7c4b-406c-a32d-09a7f97bfdc7", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
98102025-04-30T06:44:05.301ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
98112025-04-30T06:44:05.304ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "22174c4a-9728-434a-82e1-34f98c3a5cb8", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
98122025-04-30T06:44:05.304ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
98132025-04-30T06:44:05.306ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "dc249983-e4f8-43ef-a386-cbc827ebf99a", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
98142025-04-30T06:44:05.306ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
98152025-04-30T06:44:05.317ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d17fa766-f8bc-488d-95e8-0a27386e6aa0", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
98162025-04-30T06:44:05.317ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
98172025-04-30T06:44:05.322ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e90d826c-13f2-4ece-a004-20e9924edfe4", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
98182025-04-30T06:44:05.322ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
98192025-04-30T06:44:05.323ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
98202025-04-30T06:44:05.328ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "75c771b5-2028-4d5a-8ad3-1e6994561fd5", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
98212025-04-30T06:44:05.328ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
98222025-04-30T06:44:05.331ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "10bd6f6b-7762-4161-8adc-f396f5f33a4e", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
98232025-04-30T06:44:05.331ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
98242025-04-30T06:44:05.335ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "26c2ed9c-74e1-412b-b1e3-d290c1f020f4", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
98252025-04-30T06:44:05.335ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
98262025-04-30T06:44:05.339ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1aed055e-e9fc-4c46-aff5-72f478e80c53", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
98272025-04-30T06:44:05.339ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
98282025-04-30T06:44:05.341ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c19e66f3-12f4-425a-867c-3c18d30e7513", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
98292025-04-30T06:44:05.341ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
98302025-04-30T06:44:05.343ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "72935be9-b4c8-48a4-842a-417227f483e4", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
98312025-04-30T06:44:05.343ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
98322025-04-30T06:44:05.344ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2aa4690b-deea-48dc-b372-a0015fbcec23", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
98332025-04-30T06:44:05.344ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
98342025-04-30T06:44:05.345ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "38f1fa0b-6d2e-40f4-b813-2b9b763d60d5", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
98352025-04-30T06:44:05.346ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
98362025-04-30T06:44:05.346ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f6d1205e-2f91-40b1-b1f1-a4e7b28e8ba9", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
98372025-04-30T06:44:05.347ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
98382025-04-30T06:44:05.348ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5cbecde3-e82a-4bcc-b2fa-5774ac4e8ebd", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
98392025-04-30T06:44:05.348ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
98402025-04-30T06:44:05.349ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f70b5e7a-87fa-433a-afc4-d6613e34bba0", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
98412025-04-30T06:44:05.349ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
98422025-04-30T06:44:05.351ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "65bf8546-ca92-4a3f-bcc9-198e2c4e297f", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
98432025-04-30T06:44:05.352ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
98442025-04-30T06:44:05.353ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "24e64dcd-fb1d-4e7a-a09c-b60d103d6b53", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
98452025-04-30T06:44:05.356ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
98462025-04-30T06:44:05.360ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "270281ac-4372-4a96-b510-32f790a9f6ce", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
98472025-04-30T06:44:05.360ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
98482025-04-30T06:44:05.368ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "dd334a27-d166-4e2e-b270-b861bc610a9b", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
98492025-04-30T06:44:05.369ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
98502025-04-30T06:44:05.373ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7d168703-2878-4d75-8182-c13ea28b0876", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
98512025-04-30T06:44:05.374ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
98522025-04-30T06:44:05.374ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
98532025-04-30T06:44:05.374ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
98542025-04-30T06:44:05.374ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
98552025-04-30T06:44:05.374ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
98562025-04-30T06:44:05.376ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
98572025-04-30T06:44:05.380ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7e7a7143-34d1-4c84-8c1b-997b6870e0cf", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
98582025-04-30T06:44:05.381ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
98592025-04-30T06:44:05.381ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98602025-04-30T06:44:05.395ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
98612025-04-30T06:44:05.395ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
98622025-04-30T06:44:05.395ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
98632025-04-30T06:44:05.395ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
98642025-04-30T06:44:05.398ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "15f9bb7a-9a13-4ca1-9534-9c090995809a", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
98652025-04-30T06:44:05.398ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
98662025-04-30T06:44:05.398ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
98672025-04-30T06:44:05.398ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
98682025-04-30T06:44:05.399ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
98692025-04-30T06:44:05.400ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ba794e0e-9b39-49ed-b728-e7f8a4f1a9e2", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
98702025-04-30T06:44:05.400ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
98712025-04-30T06:44:05.403ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5f8512c8-a0d9-4f1f-afe4-0d3ed7ec017e", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
98722025-04-30T06:44:05.403ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
98732025-04-30T06:44:05.405ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7314e203-9c97-414f-953e-faea26a44d84", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
98742025-04-30T06:44:05.405ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
98752025-04-30T06:44:05.405ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
98762025-04-30T06:44:05.405ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
98772025-04-30T06:44:05.405ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
98782025-04-30T06:44:05.405ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
98792025-04-30T06:44:05.407ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98802025-04-30T06:44:05.412ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
98812025-04-30T06:44:05.412ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
98822025-04-30T06:44:05.412ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
98832025-04-30T06:44:05.412ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
98842025-04-30T06:44:05.412ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
98852025-04-30T06:44:05.413ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
98862025-04-30T06:44:05.425ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98872025-04-30T06:44:05.429ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
98882025-04-30T06:44:05.429ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
98892025-04-30T06:44:05.429ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
98902025-04-30T06:44:05.430ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
98912025-04-30T06:44:05.527ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
98922025-04-30T06:44:05.527ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory
98932025-04-30T06:44:05.528ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:36849 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
98942025-04-30T06:44:05.529ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:36849 req_id = b59864ed-bace-4ea1-aafa-6595d373e620 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
98952025-04-30T06:44:05.529ZINFOtest_unauthorized (dropshot): request completed latency_us = 566 local_addr = [::1]:57786 method = GET remote_addr = [::1]:36849 req_id = b59864ed-bace-4ea1-aafa-6595d373e620 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
98962025-04-30T06:44:05.530ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = http://[::1]:57786 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:57786/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "b59864ed-bace-4ea1-aafa-6595d373e620", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
98972025-04-30T06:44:05.532ZDEBGtest_unauthorized (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
98982025-04-30T06:44:05.532ZDEBGtest_unauthorized (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
98992025-04-30T06:44:05.532ZDEBGtest_unauthorized (BackgroundTasks): finished collection background_task = inventory_collection
99002025-04-30T06:44:05.535ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
99012025-04-30T06:44:05.535ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
99022025-04-30T06:44:05.535ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
99032025-04-30T06:44:05.535ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
99042025-04-30T06:44:05.536ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
99052025-04-30T06:44:05.539ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99062025-04-30T06:44:05.550ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
99072025-04-30T06:44:05.550ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
99082025-04-30T06:44:05.550ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
99092025-04-30T06:44:05.551ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
99102025-04-30T06:44:05.571ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
99112025-04-30T06:44:05.598ZWARNtest_unauthorized (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
99122025-04-30T06:44:05.599ZWARNtest_unauthorized (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
99132025-04-30T06:44:05.599ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 608.619625ms iteration = 34 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
99142025-04-30T06:44:05.599ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 35 reason = Signaled
99152025-04-30T06:44:05.599ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
99162025-04-30T06:44:05.599ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
99172025-04-30T06:44:05.599ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
99182025-04-30T06:44:05.599ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
99192025-04-30T06:44:05.600ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
99202025-04-30T06:44:05.602ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99212025-04-30T06:44:05.607ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
99222025-04-30T06:44:05.607ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
99232025-04-30T06:44:05.607ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
99242025-04-30T06:44:05.607ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
99252025-04-30T06:44:05.611ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1a8b2318-abf8-406d-a389-112ab91a0de8, 0)]
99262025-04-30T06:44:05.611ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
99272025-04-30T06:44:05.611ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 2a 95 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 *............_mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:52106 req_id = 4a7d7e7e-f854-4d18-b309-55afc7483341
99282025-04-30T06:44:05.611ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 10901,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52106 req_id = 4a7d7e7e-f854-4d18-b309-55afc7483341
99292025-04-30T06:44:05.611ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
99302025-04-30T06:44:05.612ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
99312025-04-30T06:44:05.612ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:52106 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 4a7d7e7e-f854-4d18-b309-55afc7483341
99322025-04-30T06:44:05.612ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4310, tv_nsec: 411412602 } })
99332025-04-30T06:44:05.612ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
99342025-04-30T06:44:05.612ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 06 2f 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ./..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:52136 req_id = e8e2aeaf-d042-45ee-a970-85843041ae49
99352025-04-30T06:44:05.612ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 1583,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52136 req_id = e8e2aeaf-d042-45ee-a970-85843041ae49
99362025-04-30T06:44:05.613ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
99372025-04-30T06:44:05.613ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:52136 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = e8e2aeaf-d042-45ee-a970-85843041ae49
99382025-04-30T06:44:05.693ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
99392025-04-30T06:44:05.694ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
99402025-04-30T06:44:05.694ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
99412025-04-30T06:44:05.694ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
99422025-04-30T06:44:05.694ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
99432025-04-30T06:44:05.695ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
99442025-04-30T06:44:05.697ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 19 dc 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:65058 req_id = 09d771eb-02a1-4377-8721-4247ec07fa1a
99452025-04-30T06:44:05.697ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 6620,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:65058 req_id = 09d771eb-02a1-4377-8721-4247ec07fa1a
99462025-04-30T06:44:05.697ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
99472025-04-30T06:44:05.697ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 6620, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:65058 req_id = 09d771eb-02a1-4377-8721-4247ec07fa1a
99482025-04-30T06:44:05.700ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
99492025-04-30T06:44:05.700ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
99502025-04-30T06:44:05.700ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
99512025-04-30T06:44:05.701ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99522025-04-30T06:44:05.713ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
99532025-04-30T06:44:05.713ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
99542025-04-30T06:44:05.713ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
99552025-04-30T06:44:05.714ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
99562025-04-30T06:44:05.718ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "abf6a156-b35d-491c-99d9-99d87ce0fdd1", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
99572025-04-30T06:44:05.718ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
99582025-04-30T06:44:05.718ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
99592025-04-30T06:44:05.718ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
99602025-04-30T06:44:05.719ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
99612025-04-30T06:44:05.721ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "73d09e1e-c1c3-4441-8062-99af3f3fb3d9", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
99622025-04-30T06:44:05.722ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
99632025-04-30T06:44:05.736ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5ccc9a48-93f1-47e5-925c-6aafded12fd7", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
99642025-04-30T06:44:05.737ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
99652025-04-30T06:44:05.743ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5c9b79c7-b9a6-4233-9f6f-f1c2caf20242", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
99662025-04-30T06:44:05.743ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
99672025-04-30T06:44:05.747ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bfbcb21b-5c7b-42d5-a19b-1eaad162405e", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
99682025-04-30T06:44:05.748ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
99692025-04-30T06:44:05.751ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "553d2052-187c-4d98-8b51-58431e007dce", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
99702025-04-30T06:44:05.752ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
99712025-04-30T06:44:05.755ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c175c7f3-7a10-4719-be16-28bd3d34314c", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
99722025-04-30T06:44:05.755ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
99732025-04-30T06:44:05.765ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
99742025-04-30T06:44:05.768ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "546e0479-b88b-4fd0-ac7e-2855aeefe442", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
99752025-04-30T06:44:05.768ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
99762025-04-30T06:44:05.770ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7462bf39-5bff-489a-9faf-727a0dd570e5", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
99772025-04-30T06:44:05.770ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
99782025-04-30T06:44:05.771ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ad977588-6dc0-4bc8-b37c-d3b0d34604ee", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
99792025-04-30T06:44:05.772ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
99802025-04-30T06:44:05.773ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "776d9488-cdda-48e1-a705-195456b27607", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
99812025-04-30T06:44:05.773ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
99822025-04-30T06:44:05.774ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ca99b31e-4c2f-4d25-a865-a2c479dd0b57", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
99832025-04-30T06:44:05.774ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
99842025-04-30T06:44:05.776ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a3f5a9c0-f378-45b6-a792-ebf07609c58a", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
99852025-04-30T06:44:05.777ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
99862025-04-30T06:44:05.780ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0b7f266-1b29-42b3-a7df-a6f784bb2414", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
99872025-04-30T06:44:05.780ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
99882025-04-30T06:44:05.784ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "92c166bc-ee32-4f85-8729-cb166da06779", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
99892025-04-30T06:44:05.784ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
99902025-04-30T06:44:05.787ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3d5ae210-e168-4d7c-8346-9365a9befe65", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
99912025-04-30T06:44:05.787ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
99922025-04-30T06:44:05.791ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "50d2c11f-0221-4eba-8e2d-2457686baf15", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
99932025-04-30T06:44:05.791ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
99942025-04-30T06:44:05.794ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a46ce745-890e-4db9-9811-2861cb3bd6ac", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
99952025-04-30T06:44:05.794ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
99962025-04-30T06:44:05.804ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "217c2544-f3c0-43f7-b5b5-9bdd4d4bdd1f", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
99972025-04-30T06:44:05.805ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
99982025-04-30T06:44:05.807ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0681ad36-8e16-4578-bf91-21bbcc48312e", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
99992025-04-30T06:44:05.807ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
100002025-04-30T06:44:05.808ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "770164e0-8e37-4f55-b999-0fff642f30a6", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
100012025-04-30T06:44:05.808ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
100022025-04-30T06:44:05.811ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a0716ac4-46ca-4038-a991-aa05562e07c5", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
100032025-04-30T06:44:05.811ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
100042025-04-30T06:44:05.815ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "45ddb54e-a5e2-4935-9133-4b2d743b3727", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
100052025-04-30T06:44:05.816ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
100062025-04-30T06:44:05.816ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
100072025-04-30T06:44:05.816ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
100082025-04-30T06:44:05.816ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
100092025-04-30T06:44:05.816ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
100102025-04-30T06:44:05.818ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
100112025-04-30T06:44:05.822ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5ba95dcf-e634-455e-adae-62a8e8592bb1", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
100122025-04-30T06:44:05.822ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
100132025-04-30T06:44:05.822ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100142025-04-30T06:44:05.830ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
100152025-04-30T06:44:05.830ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
100162025-04-30T06:44:05.830ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
100172025-04-30T06:44:05.832ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
100182025-04-30T06:44:05.835ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c1c1af4e-e7fb-4a47-b7ed-783b7cdc8a24", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
100192025-04-30T06:44:05.835ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
100202025-04-30T06:44:05.835ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
100212025-04-30T06:44:05.836ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
100222025-04-30T06:44:05.836ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
100232025-04-30T06:44:05.843ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "815dab8d-209c-4e21-90e7-15371187e15b", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
100242025-04-30T06:44:05.843ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
100252025-04-30T06:44:05.848ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "23f6d6f6-7192-45b6-b920-afd2fc040395", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
100262025-04-30T06:44:05.848ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
100272025-04-30T06:44:05.855ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b02dd31a-7a21-44bb-a904-e27826efc0a8", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
100282025-04-30T06:44:05.855ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
100292025-04-30T06:44:05.866ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e2728dec-219b-47d4-bfe8-c75f31b9185a", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
100302025-04-30T06:44:05.866ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
100312025-04-30T06:44:05.867ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
100322025-04-30T06:44:05.871ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "70562297-cfce-459d-83f3-131611f0a02d", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
100332025-04-30T06:44:05.872ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
100342025-04-30T06:44:05.889ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2d34e173-c0cd-451b-ad18-984be4a15876", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
100352025-04-30T06:44:05.889ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
100362025-04-30T06:44:05.892ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "51235b08-b3f3-4c38-9173-fcfb31d821c7", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
100372025-04-30T06:44:05.892ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
100382025-04-30T06:44:05.893ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fbce3578-af32-46fe-83b3-391cab53061a", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
100392025-04-30T06:44:05.894ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
100402025-04-30T06:44:05.896ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0755cff8-2016-4b51-afc7-b5a1e387f827", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
100412025-04-30T06:44:05.897ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
100422025-04-30T06:44:05.898ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "baa32107-790d-4044-bc95-55c3bd482917", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
100432025-04-30T06:44:05.898ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
100442025-04-30T06:44:05.900ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "23ea6190-b7b7-45cb-a4e5-5eb2e356abc4", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
100452025-04-30T06:44:05.901ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
100462025-04-30T06:44:05.906ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "abed9007-edbe-4902-9eff-82f969fc32cd", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
100472025-04-30T06:44:05.908ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
100482025-04-30T06:44:05.912ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6af9a677-a058-49db-8887-deff4ecca3a4", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
100492025-04-30T06:44:05.913ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
100502025-04-30T06:44:05.917ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ae40b45a-9b8d-4f78-9c38-9e94c46ff118", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
100512025-04-30T06:44:05.917ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
100522025-04-30T06:44:05.918ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
100532025-04-30T06:44:05.918ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
100542025-04-30T06:44:05.918ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
100552025-04-30T06:44:05.918ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
100562025-04-30T06:44:05.920ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
100572025-04-30T06:44:05.922ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c27a2f8f-8cd5-4c3e-b315-d46a3fba4c52", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
100582025-04-30T06:44:05.922ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
100592025-04-30T06:44:05.922ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100602025-04-30T06:44:05.929ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
100612025-04-30T06:44:05.929ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
100622025-04-30T06:44:05.929ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
100632025-04-30T06:44:05.930ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
100642025-04-30T06:44:05.934ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
100652025-04-30T06:44:05.934ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
100662025-04-30T06:44:05.934ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
100672025-04-30T06:44:05.936ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4302f853-225a-4857-840b-5cd7136bfc45", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
100682025-04-30T06:44:05.937ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
100692025-04-30T06:44:05.940ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a80a6d9d-62b7-4799-9483-8a89efcf082d", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
100702025-04-30T06:44:05.940ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
100712025-04-30T06:44:05.944ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5c7445e1-02a3-4e32-9c57-9e358a29ab66", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
100722025-04-30T06:44:05.944ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
100732025-04-30T06:44:05.947ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "66b39793-c2a9-4af9-92c8-6618aad2dbfa", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
100742025-04-30T06:44:05.947ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
100752025-04-30T06:44:05.951ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "02da9c28-77de-4d15-96c3-26f124cc2575", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
100762025-04-30T06:44:05.951ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
100772025-04-30T06:44:05.951ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
100782025-04-30T06:44:05.951ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
100792025-04-30T06:44:05.951ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
100802025-04-30T06:44:05.952ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
100812025-04-30T06:44:05.956ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100822025-04-30T06:44:05.961ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
100832025-04-30T06:44:05.962ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
100842025-04-30T06:44:05.962ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
100852025-04-30T06:44:05.962ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
100862025-04-30T06:44:05.962ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
100872025-04-30T06:44:05.963ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
100882025-04-30T06:44:05.966ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100892025-04-30T06:44:05.969ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
100902025-04-30T06:44:05.970ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
100912025-04-30T06:44:05.970ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
100922025-04-30T06:44:05.971ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
100932025-04-30T06:44:05.977ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
100942025-04-30T06:44:06.056ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
100952025-04-30T06:44:06.057ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory
100962025-04-30T06:44:06.057ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
100972025-04-30T06:44:06.057ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
100982025-04-30T06:44:06.057ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
100992025-04-30T06:44:06.057ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
101002025-04-30T06:44:06.059ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
101012025-04-30T06:44:06.059ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:54751 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
101022025-04-30T06:44:06.061ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:54751 req_id = 92ad050e-0a46-496b-88bf-f7e5286de77a sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
101032025-04-30T06:44:06.061ZINFOtest_unauthorized (dropshot): request completed latency_us = 475 local_addr = [::1]:57786 method = GET remote_addr = [::1]:54751 req_id = 92ad050e-0a46-496b-88bf-f7e5286de77a response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
101042025-04-30T06:44:06.063ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101052025-04-30T06:44:06.067ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
101062025-04-30T06:44:06.072ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
101072025-04-30T06:44:06.072ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
101082025-04-30T06:44:06.075ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
101092025-04-30T06:44:06.076ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = http://[::1]:57786 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:57786/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "92ad050e-0a46-496b-88bf-f7e5286de77a", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
101102025-04-30T06:44:06.091ZDEBGtest_unauthorized (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
101112025-04-30T06:44:06.091ZDEBGtest_unauthorized (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
101122025-04-30T06:44:06.091ZDEBGtest_unauthorized (BackgroundTasks): finished collection background_task = inventory_collection
101132025-04-30T06:44:06.095ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
101142025-04-30T06:44:06.097ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
101152025-04-30T06:44:06.097ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
101162025-04-30T06:44:06.097ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
101172025-04-30T06:44:06.101ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
101182025-04-30T06:44:06.107ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
101192025-04-30T06:44:06.107ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
101202025-04-30T06:44:06.109ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
101212025-04-30T06:44:06.110ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101222025-04-30T06:44:06.118ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
101232025-04-30T06:44:06.130ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
101242025-04-30T06:44:06.130ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
101252025-04-30T06:44:06.131ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
101262025-04-30T06:44:06.177ZDEBGtest_unauthorized (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
101272025-04-30T06:44:06.177ZDEBGtest_unauthorized (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
101282025-04-30T06:44:06.177ZDEBGtest_unauthorized (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a
101292025-04-30T06:44:06.179ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
101302025-04-30T06:44:06.190ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 2 table_name = fields_string
101312025-04-30T06:44:06.190ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
101322025-04-30T06:44:06.196ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 1 table_name = measurements_i64
101332025-04-30T06:44:06.196ZTRACtest_unauthorized (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
101342025-04-30T06:44:06.206ZDEBGtest_unauthorized (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
101352025-04-30T06:44:06.206ZDEBGtest_unauthorized (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
101362025-04-30T06:44:06.223ZTRACtest_unauthorized (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
101372025-04-30T06:44:06.225ZDEBGtest_unauthorized (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
101382025-04-30T06:44:06.225ZTRACtest_unauthorized (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
101392025-04-30T06:44:06.225ZTRACtest_unauthorized (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
101402025-04-30T06:44:06.226ZWARNtest_unauthorized (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
101412025-04-30T06:44:06.226ZWARNtest_unauthorized (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
101422025-04-30T06:44:06.226ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 627.579582ms iteration = 35 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
101432025-04-30T06:44:06.226ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 36 reason = Signaled
101442025-04-30T06:44:06.226ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
101452025-04-30T06:44:06.226ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
101462025-04-30T06:44:06.226ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
101472025-04-30T06:44:06.226ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
101482025-04-30T06:44:06.230ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
101492025-04-30T06:44:06.236ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
101502025-04-30T06:44:06.239ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101512025-04-30T06:44:06.245ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
101522025-04-30T06:44:06.245ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
101532025-04-30T06:44:06.246ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
101542025-04-30T06:44:06.247ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
101552025-04-30T06:44:06.252ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1a8b2318-abf8-406d-a389-112ab91a0de8, 0)]
101562025-04-30T06:44:06.252ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
101572025-04-30T06:44:06.253ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 20 7f 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............_mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:59898 req_id = c2ad94c5-f7b2-4797-9c95-cf9e647d3e73
101582025-04-30T06:44:06.253ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 8319,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:59898 req_id = c2ad94c5-f7b2-4797-9c95-cf9e647d3e73
101592025-04-30T06:44:06.253ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
101602025-04-30T06:44:06.254ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
101612025-04-30T06:44:06.257ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:59898 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = c2ad94c5-f7b2-4797-9c95-cf9e647d3e73
101622025-04-30T06:44:06.260ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4311, tv_nsec: 59538461 } })
101632025-04-30T06:44:06.260ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
101642025-04-30T06:44:06.261ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 58 d3 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e X...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:58511 req_id = 0299a5b7-28b2-4d36-9ebb-adb3c8820e08
101652025-04-30T06:44:06.261ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 22739,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:58511 req_id = 0299a5b7-28b2-4d36-9ebb-adb3c8820e08
101662025-04-30T06:44:06.261ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
101672025-04-30T06:44:06.263ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:58511 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 0299a5b7-28b2-4d36-9ebb-adb3c8820e08
101682025-04-30T06:44:06.361ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
101692025-04-30T06:44:06.362ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
101702025-04-30T06:44:06.362ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
101712025-04-30T06:44:06.362ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
101722025-04-30T06:44:06.362ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
101732025-04-30T06:44:06.363ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
101742025-04-30T06:44:06.363ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 2 reason = Timeout
101752025-04-30T06:44:06.363ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database
101762025-04-30T06:44:06.363ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
101772025-04-30T06:44:06.364ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
101782025-04-30T06:44:06.364ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = bfd_manager iteration = 2 reason = Timeout
101792025-04-30T06:44:06.364ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database
101802025-04-30T06:44:06.364ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
101812025-04-30T06:44:06.364ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
101822025-04-30T06:44:06.364ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = phantom_disks iteration = 2 reason = Timeout
101832025-04-30T06:44:06.364ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 3 reason = Timeout
101842025-04-30T06:44:06.364ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 42.244\u{b5}s iteration = 3 status = {"error":"task disabled"}
101852025-04-30T06:44:06.364ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 2 reason = Timeout
101862025-04-30T06:44:06.364ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory
101872025-04-30T06:44:06.364ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_zone_nat_tracker resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
101882025-04-30T06:44:06.364ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database
101892025-04-30T06:44:06.365ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
101902025-04-30T06:44:06.365ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
101912025-04-30T06:44:06.365ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 2 reason = Timeout
101922025-04-30T06:44:06.365ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet
101932025-04-30T06:44:06.365ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = switch_port_config_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
101942025-04-30T06:44:06.365ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database
101952025-04-30T06:44:06.365ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
101962025-04-30T06:44:06.366ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
101972025-04-30T06:44:06.366ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = v2p_manager iteration = 2 reason = Timeout
101982025-04-30T06:44:06.366ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
101992025-04-30T06:44:06.366ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
102002025-04-30T06:44:06.366ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
102012025-04-30T06:44:06.366ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = instance_watcher iteration = 2 reason = Timeout
102022025-04-30T06:44:06.366ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet
102032025-04-30T06:44:06.366ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = instance_watcher resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
102042025-04-30T06:44:06.366ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database
102052025-04-30T06:44:06.366ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
102062025-04-30T06:44:06.367ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
102072025-04-30T06:44:06.367ZWARNtest_unauthorized (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
102082025-04-30T06:44:06.367ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = instance_updater iteration = 2 reason = Timeout
102092025-04-30T06:44:06.367ZINFOtest_unauthorized (BackgroundTasks): background instance updater explicitly disabled background_task = instance_updater
102102025-04-30T06:44:06.367ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = instance_updater elapsed = 87.715\u{b5}s iteration = 2 status = {"destroyed_active_vmms":0,"disabled":true,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
102112025-04-30T06:44:06.367ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = vpc_route_manager iteration = 2 reason = Timeout
102122025-04-30T06:44:06.367ZINFOtest_unauthorized (BackgroundTasks): VPC route manager running background_task = vpc_route_manager
102132025-04-30T06:44:06.367ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet
102142025-04-30T06:44:06.367ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = vpc_route_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
102152025-04-30T06:44:06.368ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
102162025-04-30T06:44:06.368ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
102172025-04-30T06:44:06.368ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
102182025-04-30T06:44:06.368ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 9d 37 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c .7..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:52018 req_id = 6570b2d0-6173-4d56-be85-19479447593d
102192025-04-30T06:44:06.368ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 40247,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52018 req_id = 6570b2d0-6173-4d56-be85-19479447593d
102202025-04-30T06:44:06.368ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
102212025-04-30T06:44:06.368ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 40247, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:52018 req_id = 6570b2d0-6173-4d56-be85-19479447593d
102222025-04-30T06:44:06.369ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
102232025-04-30T06:44:06.369ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
102242025-04-30T06:44:06.369ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
102252025-04-30T06:44:06.370ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102262025-04-30T06:44:06.375ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
102272025-04-30T06:44:06.375ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
102282025-04-30T06:44:06.375ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
102292025-04-30T06:44:06.376ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
102302025-04-30T06:44:06.377ZTRACtest_unauthorized (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
102312025-04-30T06:44:06.389ZTRACtest_unauthorized (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
102322025-04-30T06:44:06.390ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 24.825473ms iteration = 2 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
102332025-04-30T06:44:06.391ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102342025-04-30T06:44:06.397ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
102352025-04-30T06:44:06.397ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database
102362025-04-30T06:44:06.397ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
102372025-04-30T06:44:06.398ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
102382025-04-30T06:44:06.398ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102392025-04-30T06:44:06.402ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
102402025-04-30T06:44:06.402ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database
102412025-04-30T06:44:06.402ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
102422025-04-30T06:44:06.402ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
102432025-04-30T06:44:06.409ZTRACtest_unauthorized (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 31 d7 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 1............_de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:57314 req_id = af670e12-d630-4f95-882e-5cf610a35609
102442025-04-30T06:44:06.410ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 12759,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:57314 req_id = af670e12-d630-4f95-882e-5cf610a35609
102452025-04-30T06:44:06.410ZDEBGtest_unauthorized (store): query key key = _dendrite._tcp
102462025-04-30T06:44:06.410ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
102472025-04-30T06:44:06.411ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:57314 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 44264, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = af670e12-d630-4f95-882e-5cf610a35609
102482025-04-30T06:44:06.411ZTRACtest_unauthorized (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: db 87 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 ............._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:63078 req_id = fa5c8f27-a731-4a7e-bc31-41a665df0ffa
102492025-04-30T06:44:06.411ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 56199,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63078 req_id = fa5c8f27-a731-4a7e-bc31-41a665df0ffa
102502025-04-30T06:44:06.411ZDEBGtest_unauthorized (store): query key key = _dendrite._tcp
102512025-04-30T06:44:06.411ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
102522025-04-30T06:44:06.412ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:63078 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 44264, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = fa5c8f27-a731-4a7e-bc31-41a665df0ffa
102532025-04-30T06:44:06.412ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102542025-04-30T06:44:06.426ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
102552025-04-30T06:44:06.426ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database
102562025-04-30T06:44:06.426ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
102572025-04-30T06:44:06.427ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
102582025-04-30T06:44:06.450ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102592025-04-30T06:44:06.455ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
102602025-04-30T06:44:06.455ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet
102612025-04-30T06:44:06.466ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = vpc_route_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
102622025-04-30T06:44:06.466ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
102632025-04-30T06:44:06.468ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
102642025-04-30T06:44:06.468ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
102652025-04-30T06:44:06.468ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
102662025-04-30T06:44:06.468ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
102672025-04-30T06:44:06.469ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
102682025-04-30T06:44:06.474ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database
102692025-04-30T06:44:06.474ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
102702025-04-30T06:44:06.476ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
102712025-04-30T06:44:06.477ZTRACtest_unauthorized (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
102722025-04-30T06:44:06.483ZDEBGtest_unauthorized (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 44264, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4311, tv_nsec: 281756496 } })
102732025-04-30T06:44:06.483ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
102742025-04-30T06:44:06.484ZDEBGtest_unauthorized (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 44264, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4311, tv_nsec: 282941567 } })
102752025-04-30T06:44:06.484ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
102762025-04-30T06:44:06.484ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "6307004b-f4d5-4b00-8612-4e6d1dd13d74", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
102772025-04-30T06:44:06.484ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
102782025-04-30T06:44:06.493ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102792025-04-30T06:44:06.498ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
102802025-04-30T06:44:06.498ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
102812025-04-30T06:44:06.498ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
102822025-04-30T06:44:06.499ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
102832025-04-30T06:44:06.499ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet
102842025-04-30T06:44:06.499ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = v2p_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
102852025-04-30T06:44:06.499ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
102862025-04-30T06:44:06.499ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
102872025-04-30T06:44:06.500ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
102882025-04-30T06:44:06.513ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
102892025-04-30T06:44:06.513ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
102902025-04-30T06:44:06.514ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
102912025-04-30T06:44:06.515ZTRACtest_unauthorized (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 5e a5 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 ^............_de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:65528 req_id = dfeb1d27-62a3-41e5-a2af-87003481f576
102922025-04-30T06:44:06.515ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 24229,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:65528 req_id = dfeb1d27-62a3-41e5-a2af-87003481f576
102932025-04-30T06:44:06.515ZDEBGtest_unauthorized (store): query key key = _dendrite._tcp
102942025-04-30T06:44:06.515ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
102952025-04-30T06:44:06.516ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:65528 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 44264, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = dfeb1d27-62a3-41e5-a2af-87003481f576
102962025-04-30T06:44:06.518ZDEBGtest_unauthorized (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 44264, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4311, tv_nsec: 317187370 } })
102972025-04-30T06:44:06.518ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
102982025-04-30T06:44:06.518ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 4a 3c 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e J<..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:56304 req_id = 7e47d07b-430c-4475-a81d-28664436262b
102992025-04-30T06:44:06.518ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 19004,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:56304 req_id = 7e47d07b-430c-4475-a81d-28664436262b
103002025-04-30T06:44:06.518ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
103012025-04-30T06:44:06.518ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:56304 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 7e47d07b-430c-4475-a81d-28664436262b
103022025-04-30T06:44:06.518ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 78 51 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e xQ..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:62032 req_id = 8b64062e-1b56-42b5-bca8-f7eb908fd4d8
103032025-04-30T06:44:06.518ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 30801,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62032 req_id = 8b64062e-1b56-42b5-bca8-f7eb908fd4d8
103042025-04-30T06:44:06.518ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
103052025-04-30T06:44:06.519ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:62032 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 8b64062e-1b56-42b5-bca8-f7eb908fd4d8
103062025-04-30T06:44:06.525ZINFOtest_unauthorized (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector
103072025-04-30T06:44:06.608ZINFOtest_unauthorized (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector zone_address = ::1
103082025-04-30T06:44:06.608ZDEBGtest_unauthorized (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[::1]:12225/local/switch-id
103092025-04-30T06:44:06.608ZDEBGtest_unauthorized (MgsClient): client response background_task = nat_v4_garbage_collector result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
103102025-04-30T06:44:06.608ZWARNtest_unauthorized (BackgroundTasks): failed to identify switch slot for dendrite background_task = nat_v4_garbage_collector reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
103112025-04-30T06:44:06.610ZWARNtest_unauthorized (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = nat_v4_garbage_collector
103122025-04-30T06:44:06.611ZINFOtest_unauthorized (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager
103132025-04-30T06:44:06.716ZINFOtest_unauthorized (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager zone_address = ::1
103142025-04-30T06:44:06.716ZDEBGtest_unauthorized (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[::1]:12225/local/switch-id
103152025-04-30T06:44:06.718ZDEBGtest_unauthorized (MgsClient): client response background_task = bfd_manager result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
103162025-04-30T06:44:06.718ZWARNtest_unauthorized (BackgroundTasks): failed to identify switch slot for dendrite background_task = bfd_manager reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
103172025-04-30T06:44:06.723ZWARNtest_unauthorized (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = bfd_manager
103182025-04-30T06:44:06.724ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103192025-04-30T06:44:06.771ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
103202025-04-30T06:44:06.771ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet
103212025-04-30T06:44:06.771ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = v2p_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
103222025-04-30T06:44:06.771ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
103232025-04-30T06:44:06.771ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
103242025-04-30T06:44:06.772ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
103252025-04-30T06:44:06.773ZINFOtest_unauthorized (BackgroundTasks): all instance checks complete background_task = instance_watcher pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
103262025-04-30T06:44:06.773ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 407.138266ms iteration = 2 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
103272025-04-30T06:44:06.872ZINFOtest_unauthorized (BackgroundTasks): VPC route manager sled b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager
103282025-04-30T06:44:06.872ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager body = None method = GET uri = http://[::1]:57786/vpc-routes
103292025-04-30T06:44:06.874ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103302025-04-30T06:44:06.887ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
103312025-04-30T06:44:06.887ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
103322025-04-30T06:44:06.887ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
103332025-04-30T06:44:06.888ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
103342025-04-30T06:44:06.890ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:59154 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
103352025-04-30T06:44:06.891ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 20 86 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:61464 req_id = f2582424-c36b-4447-9418-eb0ce28a22da
103362025-04-30T06:44:06.892ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 8326,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:61464 req_id = f2582424-c36b-4447-9418-eb0ce28a22da
103372025-04-30T06:44:06.892ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
103382025-04-30T06:44:06.892ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:61464 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = f2582424-c36b-4447-9418-eb0ce28a22da
103392025-04-30T06:44:06.908ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:59154 req_id = 23d2da04-72be-4192-9347-86fb2680e5bf sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc-routes
103402025-04-30T06:44:06.908ZINFOtest_unauthorized (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc
103412025-04-30T06:44:06.973ZINFOtest_unauthorized (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc zone_address = ::1
103422025-04-30T06:44:06.973ZDEBGtest_unauthorized (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc uri = http://[::1]:12225/local/switch-id
103432025-04-30T06:44:06.973ZDEBGtest_unauthorized (MgsClient): client response background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
103442025-04-30T06:44:06.973ZWARNtest_unauthorized (BackgroundTasks): failed to identify switch slot for dendrite background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
103452025-04-30T06:44:06.975ZWARNtest_unauthorized (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc
103462025-04-30T06:44:06.977ZINFOtest_unauthorized (dropshot): request completed latency_us = 68906 local_addr = [::1]:57786 method = GET remote_addr = [::1]:59154 req_id = 23d2da04-72be-4192-9347-86fb2680e5bf response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc-routes
103472025-04-30T06:44:07.046ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = v2p_manager body = None method = GET uri = http://[::1]:57786/v2p
103482025-04-30T06:44:07.049ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:40929 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
103492025-04-30T06:44:07.050ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9be16ae8-7799-4ece-a678-f9a5691dd9da", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
103502025-04-30T06:44:07.050ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
103512025-04-30T06:44:07.050ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager result = Ok(Response { url: "http://[::1]:57786/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "23d2da04-72be-4192-9347-86fb2680e5bf", "content-length": "2", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
103522025-04-30T06:44:07.050ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
103532025-04-30T06:44:07.050ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
103542025-04-30T06:44:07.052ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
103552025-04-30T06:44:07.073ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:40929 req_id = 9f4bc710-0c15-4f2d-8323-822ddffa8ba3 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /v2p
103562025-04-30T06:44:07.076ZINFOtest_unauthorized (dropshot): request completed latency_us = 2474 local_addr = [::1]:57786 method = GET remote_addr = [::1]:40929 req_id = 9f4bc710-0c15-4f2d-8323-822ddffa8ba3 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /v2p
103572025-04-30T06:44:07.080ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = v2p_manager result = Ok(Response { url: "http://[::1]:57786/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "9f4bc710-0c15-4f2d-8323-822ddffa8ba3", "content-length": "2", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
103582025-04-30T06:44:07.081ZINFOtest_unauthorized (BackgroundTasks): found opte v2p mappings background_task = v2p_manager interfaces = {} sled = sim-b6d65341
103592025-04-30T06:44:07.081ZINFOtest_unauthorized (BackgroundTasks): v2p mappings to delete background_task = v2p_manager mappings = [] sled = sim-b6d65341
103602025-04-30T06:44:07.081ZINFOtest_unauthorized (BackgroundTasks): v2p mappings to add background_task = v2p_manager mappings = [] sled = sim-b6d65341
103612025-04-30T06:44:07.083ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 717.195134ms iteration = 2 status = {}
103622025-04-30T06:44:07.085ZINFOtest_unauthorized (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager sled = sim-b6d65341
103632025-04-30T06:44:07.085ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[::1]:57786/eip-gateways
103642025-04-30T06:44:07.087ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = PUT remote_addr = [::1]:59154 req_id = 5844aa66-8654-4112-a8a5-927dbab3073c sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /eip-gateways
103652025-04-30T06:44:07.090ZINFOtest_unauthorized (dropshot): request completed latency_us = 2671 local_addr = [::1]:57786 method = PUT remote_addr = [::1]:59154 req_id = 5844aa66-8654-4112-a8a5-927dbab3073c response_code = 204 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /eip-gateways
103662025-04-30T06:44:07.094ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager result = Ok(Response { url: "http://[::1]:57786/eip-gateways", status: 204, headers: {"x-request-id": "5844aa66-8654-4112-a8a5-927dbab3073c", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
103672025-04-30T06:44:07.099ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 731.559164ms iteration = 2 status = {}
103682025-04-30T06:44:07.101ZINFOtest_unauthorized (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
103692025-04-30T06:44:07.101ZDEBGtest_unauthorized (oximeter): client request body = None method = GET uri = http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
103702025-04-30T06:44:07.103ZTRACtest_unauthorized (dropshot_internal): incoming request local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 1b7f2844-85b0-4592-8ad2-d9e0f95b77b1 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
103712025-04-30T06:44:07.106ZTRACtest_unauthorized (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 1b7f2844-85b0-4592-8ad2-d9e0f95b77b1 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
103722025-04-30T06:44:07.106ZDEBGtest_unauthorized (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 1b7f2844-85b0-4592-8ad2-d9e0f95b77b1 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
103732025-04-30T06:44:07.109ZDEBGtest_unauthorized (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 1b7f2844-85b0-4592-8ad2-d9e0f95b77b1 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
103742025-04-30T06:44:07.114ZINFOtest_unauthorized (dropshot_internal): request completed latency_us = 11057 local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 1b7f2844-85b0-4592-8ad2-d9e0f95b77b1 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
103752025-04-30T06:44:07.120ZDEBGtest_unauthorized (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "1b7f2844-85b0-4592-8ad2-d9e0f95b77b1", "content-length": "405", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
103762025-04-30T06:44:07.120ZDEBGtest_unauthorized (oximeter): client request body = None method = GET uri = http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
103772025-04-30T06:44:07.127ZTRACtest_unauthorized (dropshot_internal): incoming request local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 158439d1-5b2e-425e-8b7c-0e7805a6a643 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
103782025-04-30T06:44:07.128ZTRACtest_unauthorized (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 158439d1-5b2e-425e-8b7c-0e7805a6a643 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
103792025-04-30T06:44:07.128ZDEBGtest_unauthorized (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 158439d1-5b2e-425e-8b7c-0e7805a6a643 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
103802025-04-30T06:44:07.129ZDEBGtest_unauthorized (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 158439d1-5b2e-425e-8b7c-0e7805a6a643 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
103812025-04-30T06:44:07.129ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database
103822025-04-30T06:44:07.129ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
103832025-04-30T06:44:07.130ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
103842025-04-30T06:44:07.149ZINFOtest_unauthorized (dropshot_internal): request completed latency_us = 21721 local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 158439d1-5b2e-425e-8b7c-0e7805a6a643 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
103852025-04-30T06:44:07.152ZDEBGtest_unauthorized (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "158439d1-5b2e-425e-8b7c-0e7805a6a643", "content-length": "29", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
103862025-04-30T06:44:07.152ZTRACtest_unauthorized (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 8d752adb-08b9-405f-a11b-66019bf447b5
103872025-04-30T06:44:07.152ZTRACtest_unauthorized (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
103882025-04-30T06:44:07.152ZINFOtest_unauthorized (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
103892025-04-30T06:44:07.157ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b6d65341-167c-41df-9b5c-41cded99c229, lookup_type: ById(b6d65341-167c-41df-9b5c-41cded99c229) }
103902025-04-30T06:44:07.157ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_zone_nat_tracker resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
103912025-04-30T06:44:07.157ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database
103922025-04-30T06:44:07.157ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
103932025-04-30T06:44:07.157ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
103942025-04-30T06:44:07.168ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103952025-04-30T06:44:07.185ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b6d65341-167c-41df-9b5c-41cded99c229, lookup_type: ById(b6d65341-167c-41df-9b5c-41cded99c229) } result = Ok(())
103962025-04-30T06:44:07.187ZERROtest_unauthorized (BackgroundTasks): ipv6 addresses for service zone nat not implemented background_task = service_zone_nat_tracker
103972025-04-30T06:44:07.187ZERROtest_unauthorized (BackgroundTasks): generated config for fewer than the minimum allowed number of dns zones background_task = service_zone_nat_tracker
103982025-04-30T06:44:07.188ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 823.843655ms iteration = 2 status = {"error":"generated config for fewer than the minimum allowed number of dns zones"}
103992025-04-30T06:44:07.191ZDEBGtest_unauthorized (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
104002025-04-30T06:44:07.198ZDEBGtest_unauthorized (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
104012025-04-30T06:44:07.200ZDEBGtest_unauthorized (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a
104022025-04-30T06:44:07.200ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
104032025-04-30T06:44:07.205ZDEBGtest_unauthorized (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
104042025-04-30T06:44:07.205ZDEBGtest_unauthorized (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
104052025-04-30T06:44:07.206ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
104062025-04-30T06:44:07.209ZTRACtest_unauthorized (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
104072025-04-30T06:44:07.210ZDEBGtest_unauthorized (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
104082025-04-30T06:44:07.210ZTRACtest_unauthorized (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
104092025-04-30T06:44:07.210ZTRACtest_unauthorized (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
104102025-04-30T06:44:07.210ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b3ec431d-bf91-4dbd-b1b4-01fd58dfe0cf", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
104112025-04-30T06:44:07.210ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
104122025-04-30T06:44:07.211ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 2 table_name = fields_string
104132025-04-30T06:44:07.211ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
104142025-04-30T06:44:07.214ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 1 table_name = measurements_i64
104152025-04-30T06:44:07.214ZTRACtest_unauthorized (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
104162025-04-30T06:44:07.215ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f585f28c-b158-451f-8566-fb9f2042b9a5", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
104172025-04-30T06:44:07.215ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
104182025-04-30T06:44:07.219ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5d183152-2103-450a-8df1-4d747adb8f83", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
104192025-04-30T06:44:07.219ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
104202025-04-30T06:44:07.223ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7ab47f8c-08be-4453-a4f5-83712433f42a", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
104212025-04-30T06:44:07.223ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
104222025-04-30T06:44:07.229ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "58a3da00-42ce-433c-88b7-91189f8707fb", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
104232025-04-30T06:44:07.229ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
104242025-04-30T06:44:07.234ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "45f21ec8-4d20-4065-976d-e1651e14d79c", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
104252025-04-30T06:44:07.235ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
104262025-04-30T06:44:07.236ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4e071303-fbf6-4764-9787-8e39ff3f560c", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
104272025-04-30T06:44:07.237ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
104282025-04-30T06:44:07.239ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "aabeabf6-f158-4548-a092-adf23fbc3a59", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
104292025-04-30T06:44:07.239ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
104302025-04-30T06:44:07.240ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "85b1edf4-5c42-43f8-9134-ce393e31b9ec", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
104312025-04-30T06:44:07.241ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
104322025-04-30T06:44:07.242ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d8a305bf-b162-4c0d-9f95-b733f2654553", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
104332025-04-30T06:44:07.243ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
104342025-04-30T06:44:07.245ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a7e444ab-ab2e-4817-969a-56a7cbee66aa", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
104352025-04-30T06:44:07.246ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
104362025-04-30T06:44:07.249ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "62387f9e-42f7-4a98-b359-493b53b2b559", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
104372025-04-30T06:44:07.249ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
104382025-04-30T06:44:07.255ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f19f18a6-1816-46fc-a7f5-2ca5630ff4f6", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
104392025-04-30T06:44:07.256ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
104402025-04-30T06:44:07.257ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
104412025-04-30T06:44:07.257ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
104422025-04-30T06:44:07.257ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
104432025-04-30T06:44:07.257ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
104442025-04-30T06:44:07.259ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
104452025-04-30T06:44:07.264ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104462025-04-30T06:44:07.274ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
104472025-04-30T06:44:07.274ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
104482025-04-30T06:44:07.274ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
104492025-04-30T06:44:07.275ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
104502025-04-30T06:44:07.278ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
104512025-04-30T06:44:07.278ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
104522025-04-30T06:44:07.279ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
104532025-04-30T06:44:07.281ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "eed33336-44ef-497b-9ff3-217454b55002", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
104542025-04-30T06:44:07.281ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
104552025-04-30T06:44:07.292ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d0303281-e2de-4cc4-8f12-e43b536b3f77", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
104562025-04-30T06:44:07.292ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
104572025-04-30T06:44:07.300ZWARNtest_unauthorized (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
104582025-04-30T06:44:07.306ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "15c5d69c-ca94-411b-97dc-aace004ac818", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
104592025-04-30T06:44:07.307ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
104602025-04-30T06:44:07.312ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "00af07b8-1395-431b-ab2b-8bc1ee58893c", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
104612025-04-30T06:44:07.313ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
104622025-04-30T06:44:07.317ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "65f59e04-326e-439b-9fd5-86e85d55afbf", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
104632025-04-30T06:44:07.317ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
104642025-04-30T06:44:07.320ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "09bd59e5-77d6-4f8c-bee5-c94167a2c5da", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
104652025-04-30T06:44:07.321ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
104662025-04-30T06:44:07.323ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5dc6a9b6-8d3b-483a-8b46-cf6395d41e61", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
104672025-04-30T06:44:07.324ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
104682025-04-30T06:44:07.327ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
104692025-04-30T06:44:07.327ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6499df59-3349-4f1a-866e-b3ffaa4e4518", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
104702025-04-30T06:44:07.328ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
104712025-04-30T06:44:07.331ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "612c8663-0438-4a44-8d3d-20d7e910bfdc", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
104722025-04-30T06:44:07.332ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
104732025-04-30T06:44:07.335ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "49cc339a-9aa6-49bb-8d71-5b2fb2a74f13", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
104742025-04-30T06:44:07.335ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
104752025-04-30T06:44:07.340ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f6c3d0d4-1663-4e18-9a53-b974b04f4a5e", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
104762025-04-30T06:44:07.340ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
104772025-04-30T06:44:07.344ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "af296937-e9fc-4b22-acf7-d189dce2b5aa", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
104782025-04-30T06:44:07.344ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
104792025-04-30T06:44:07.348ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "28551c5d-8e4b-43ae-8372-fbeada7a544e", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
104802025-04-30T06:44:07.348ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
104812025-04-30T06:44:07.352ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b25825ab-7286-4864-9c26-cb956f1b03d0", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
104822025-04-30T06:44:07.352ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
104832025-04-30T06:44:07.357ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "85aa4bd3-f7c2-4fb7-8133-c999e912af93", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
104842025-04-30T06:44:07.359ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
104852025-04-30T06:44:07.363ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5d0c846c-4c2a-4fd2-a6b8-74cef27f24dc", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
104862025-04-30T06:44:07.363ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
104872025-04-30T06:44:07.365ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "dbad13ff-ec01-4f26-b69b-28b4e06e34d5", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
104882025-04-30T06:44:07.365ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
104892025-04-30T06:44:07.373ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "26643a93-69f3-4459-9ac9-2450fcf63ee6", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
104902025-04-30T06:44:07.374ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
104912025-04-30T06:44:07.375ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0263d8f2-2da9-4b69-b667-94d29ba9c10c", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
104922025-04-30T06:44:07.375ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
104932025-04-30T06:44:07.378ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
104942025-04-30T06:44:07.378ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
104952025-04-30T06:44:07.378ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
104962025-04-30T06:44:07.378ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
104972025-04-30T06:44:07.380ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
104982025-04-30T06:44:07.381ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "24c50ae7-c73b-43df-bf7f-f2c4bbc72c35", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
104992025-04-30T06:44:07.381ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
105002025-04-30T06:44:07.394ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105012025-04-30T06:44:07.403ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
105022025-04-30T06:44:07.403ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
105032025-04-30T06:44:07.403ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
105042025-04-30T06:44:07.404ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
105052025-04-30T06:44:07.410ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
105062025-04-30T06:44:07.410ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
105072025-04-30T06:44:07.412ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
105082025-04-30T06:44:07.413ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7906b8ba-6e46-4db9-938a-e8cdf35adc16", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
105092025-04-30T06:44:07.414ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
105102025-04-30T06:44:07.417ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2bfb36dc-63b3-44cd-9508-a03e4c742de2", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
105112025-04-30T06:44:07.417ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
105122025-04-30T06:44:07.422ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4e1d2ec4-44cd-4001-8959-5361bc817d9e", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
105132025-04-30T06:44:07.422ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
105142025-04-30T06:44:07.425ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "90d19df8-16d0-4cd5-a812-ef83843ec777", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
105152025-04-30T06:44:07.425ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
105162025-04-30T06:44:07.427ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fdcb2f9a-1c13-49c0-aedb-4873426f8921", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
105172025-04-30T06:44:07.428ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
105182025-04-30T06:44:07.433ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0b84b56a-7152-4d70-b217-28eab4245f6d", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
105192025-04-30T06:44:07.433ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
105202025-04-30T06:44:07.437ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "053ce94a-7293-41fd-9ace-c80497b49bf4", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
105212025-04-30T06:44:07.439ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
105222025-04-30T06:44:07.440ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d38e7428-c82a-4263-8a2a-3d0d4e76d34c", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
105232025-04-30T06:44:07.440ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
105242025-04-30T06:44:07.442ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5a07cca5-3e13-46a0-8930-161a262279ea", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
105252025-04-30T06:44:07.442ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
105262025-04-30T06:44:07.443ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f10009c0-064b-4253-b401-3d974ac1f9f0", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
105272025-04-30T06:44:07.443ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
105282025-04-30T06:44:07.443ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
105292025-04-30T06:44:07.443ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
105302025-04-30T06:44:07.443ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
105312025-04-30T06:44:07.444ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
105322025-04-30T06:44:07.447ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105332025-04-30T06:44:07.453ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
105342025-04-30T06:44:07.453ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
105352025-04-30T06:44:07.453ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
105362025-04-30T06:44:07.453ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
105372025-04-30T06:44:07.453ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
105382025-04-30T06:44:07.454ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
105392025-04-30T06:44:07.459ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105402025-04-30T06:44:07.465ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
105412025-04-30T06:44:07.465ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
105422025-04-30T06:44:07.465ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
105432025-04-30T06:44:07.465ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
105442025-04-30T06:44:07.466ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
105452025-04-30T06:44:07.527ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
105462025-04-30T06:44:07.527ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory
105472025-04-30T06:44:07.528ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
105482025-04-30T06:44:07.528ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
105492025-04-30T06:44:07.528ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
105502025-04-30T06:44:07.528ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
105512025-04-30T06:44:07.530ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
105522025-04-30T06:44:07.530ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:45666 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
105532025-04-30T06:44:07.533ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:45666 req_id = a5de36f1-aea5-4e48-bbc3-fdbba0876002 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
105542025-04-30T06:44:07.534ZINFOtest_unauthorized (dropshot): request completed latency_us = 532 local_addr = [::1]:57786 method = GET remote_addr = [::1]:45666 req_id = a5de36f1-aea5-4e48-bbc3-fdbba0876002 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
105552025-04-30T06:44:07.534ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = http://[::1]:57786 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:57786/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "a5de36f1-aea5-4e48-bbc3-fdbba0876002", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
105562025-04-30T06:44:07.537ZDEBGtest_unauthorized (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
105572025-04-30T06:44:07.537ZDEBGtest_unauthorized (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
105582025-04-30T06:44:07.537ZDEBGtest_unauthorized (BackgroundTasks): finished collection background_task = inventory_collection
105592025-04-30T06:44:07.542ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
105602025-04-30T06:44:07.542ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
105612025-04-30T06:44:07.542ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
105622025-04-30T06:44:07.542ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
105632025-04-30T06:44:07.543ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
105642025-04-30T06:44:07.545ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105652025-04-30T06:44:07.549ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
105662025-04-30T06:44:07.549ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
105672025-04-30T06:44:07.549ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
105682025-04-30T06:44:07.549ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
105692025-04-30T06:44:07.552ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105702025-04-30T06:44:07.555ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
105712025-04-30T06:44:07.555ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
105722025-04-30T06:44:07.555ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
105732025-04-30T06:44:07.556ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
105742025-04-30T06:44:07.556ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
105752025-04-30T06:44:07.556ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
105762025-04-30T06:44:07.557ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
105772025-04-30T06:44:07.623ZWARNtest_unauthorized (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
105782025-04-30T06:44:07.627ZWARNtest_unauthorized (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
105792025-04-30T06:44:07.627ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 1.400821546s iteration = 36 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
105802025-04-30T06:44:07.627ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 37 reason = Signaled
105812025-04-30T06:44:07.627ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
105822025-04-30T06:44:07.627ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
105832025-04-30T06:44:07.627ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
105842025-04-30T06:44:07.627ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
105852025-04-30T06:44:07.628ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
105862025-04-30T06:44:07.633ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105872025-04-30T06:44:07.639ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
105882025-04-30T06:44:07.639ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
105892025-04-30T06:44:07.639ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
105902025-04-30T06:44:07.640ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
105912025-04-30T06:44:07.644ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1a8b2318-abf8-406d-a389-112ab91a0de8, 0)]
105922025-04-30T06:44:07.644ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
105932025-04-30T06:44:07.646ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 9f 88 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:62303 req_id = fb1510b3-6f1e-4f6e-975f-94e3d17d0585
105942025-04-30T06:44:07.646ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 40840,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62303 req_id = fb1510b3-6f1e-4f6e-975f-94e3d17d0585
105952025-04-30T06:44:07.646ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
105962025-04-30T06:44:07.646ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
105972025-04-30T06:44:07.646ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:62303 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = fb1510b3-6f1e-4f6e-975f-94e3d17d0585
105982025-04-30T06:44:07.648ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
105992025-04-30T06:44:07.648ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4312, tv_nsec: 447563771 } })
106002025-04-30T06:44:07.648ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
106012025-04-30T06:44:07.649ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: f8 74 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .t..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:59631 req_id = 49712367-ddc5-4bfe-9658-b138e23571c2
106022025-04-30T06:44:07.649ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 63604,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:59631 req_id = 49712367-ddc5-4bfe-9658-b138e23571c2
106032025-04-30T06:44:07.649ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
106042025-04-30T06:44:07.649ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:59631 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 49712367-ddc5-4bfe-9658-b138e23571c2
106052025-04-30T06:44:07.709ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
106062025-04-30T06:44:07.709ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
106072025-04-30T06:44:07.709ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
106082025-04-30T06:44:07.709ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
106092025-04-30T06:44:07.709ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
106102025-04-30T06:44:07.710ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
106112025-04-30T06:44:07.711ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 8a a1 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:57716 req_id = 8ceed81c-86dd-4d36-8c35-67584c623c35
106122025-04-30T06:44:07.711ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 35489,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:57716 req_id = 8ceed81c-86dd-4d36-8c35-67584c623c35
106132025-04-30T06:44:07.712ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
106142025-04-30T06:44:07.712ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 35489, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:57716 req_id = 8ceed81c-86dd-4d36-8c35-67584c623c35
106152025-04-30T06:44:07.712ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
106162025-04-30T06:44:07.712ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
106172025-04-30T06:44:07.712ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
106182025-04-30T06:44:07.713ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106192025-04-30T06:44:07.721ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
106202025-04-30T06:44:07.721ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
106212025-04-30T06:44:07.721ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
106222025-04-30T06:44:07.721ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
106232025-04-30T06:44:07.724ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
106242025-04-30T06:44:07.724ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
106252025-04-30T06:44:07.725ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
106262025-04-30T06:44:07.725ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "651414c6-a7c1-46fe-baf3-f47069339d32", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
106272025-04-30T06:44:07.725ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
106282025-04-30T06:44:07.728ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "12a328cc-45f8-4d25-8537-b01bf01f5f23", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
106292025-04-30T06:44:07.728ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
106302025-04-30T06:44:07.731ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "dec7a96e-780c-4355-94a2-9a299e65d9d7", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
106312025-04-30T06:44:07.731ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
106322025-04-30T06:44:07.734ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e2a2845a-2017-4778-a780-026fa13bc970", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
106332025-04-30T06:44:07.734ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
106342025-04-30T06:44:07.737ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ae2875da-12d8-4d1c-8a2e-1c3b007c165b", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
106352025-04-30T06:44:07.737ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
106362025-04-30T06:44:07.741ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c3b870a2-d299-4dcc-a39b-d43c49c9a55a", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
106372025-04-30T06:44:07.741ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
106382025-04-30T06:44:07.749ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a85e6bd9-0d8d-4a68-b221-c5e0822c9582", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
106392025-04-30T06:44:07.750ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
106402025-04-30T06:44:07.757ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "15cc6b3d-d9dd-4e4a-816d-f1236b3fc91a", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
106412025-04-30T06:44:07.757ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
106422025-04-30T06:44:07.759ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2000c6c0-3ec7-4aa0-8d2e-93c1c3cf6334", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
106432025-04-30T06:44:07.759ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
106442025-04-30T06:44:07.761ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "89b68f3b-d4ae-422a-9f24-25697d087fb6", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
106452025-04-30T06:44:07.761ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
106462025-04-30T06:44:07.764ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "917fcc8e-e59e-4d81-bad3-f7680b6dce11", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
106472025-04-30T06:44:07.764ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
106482025-04-30T06:44:07.766ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8e02161c-a7ad-4acd-8e9e-af5654678da0", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
106492025-04-30T06:44:07.766ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
106502025-04-30T06:44:07.767ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "35bbb758-3025-46b0-9b63-cfafb35c317f", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
106512025-04-30T06:44:07.767ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
106522025-04-30T06:44:07.772ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
106532025-04-30T06:44:07.776ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a54b4c65-a3b7-4060-a985-5b8407ed8f6b", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
106542025-04-30T06:44:07.776ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
106552025-04-30T06:44:07.778ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cfdf64b6-3c7f-48fb-8155-6fc300a62558", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
106562025-04-30T06:44:07.779ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
106572025-04-30T06:44:07.781ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "38bcf530-daf7-4b96-9498-7a20ee37988a", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
106582025-04-30T06:44:07.781ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
106592025-04-30T06:44:07.783ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7a9f5b30-9719-47aa-998f-8fc9c75c455b", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
106602025-04-30T06:44:07.783ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
106612025-04-30T06:44:07.784ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e3116808-4ffe-4c17-b9e4-3d0308665952", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
106622025-04-30T06:44:07.785ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
106632025-04-30T06:44:07.787ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1370816c-ccd9-403c-a74a-101a3266ac20", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
106642025-04-30T06:44:07.788ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
106652025-04-30T06:44:07.789ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "71b6ea49-7975-4342-bb40-ea789d853c4d", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
106662025-04-30T06:44:07.789ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
106672025-04-30T06:44:07.790ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4ffe2442-e22a-44eb-b7dc-140c368d80c6", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
106682025-04-30T06:44:07.790ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
106692025-04-30T06:44:07.791ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e093eef8-04f4-4c19-a4e1-76665b2e472d", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
106702025-04-30T06:44:07.791ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
106712025-04-30T06:44:07.792ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7e445192-bfea-4558-ba45-89b48f45f06c", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
106722025-04-30T06:44:07.792ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
106732025-04-30T06:44:07.794ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "00b4a129-e3e5-4c98-a84b-7f5e584ba1b8", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
106742025-04-30T06:44:07.794ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
106752025-04-30T06:44:07.797ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "607ab5d7-58ff-42d5-9116-173d92fed7fc", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
106762025-04-30T06:44:07.797ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
106772025-04-30T06:44:07.799ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b6e86565-ff62-4f76-b13b-08c1f9ab23d5", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
106782025-04-30T06:44:07.799ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
106792025-04-30T06:44:07.801ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "76ff0abd-0708-4198-9214-0effb190c60d", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
106802025-04-30T06:44:07.801ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
106812025-04-30T06:44:07.803ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a8e48d35-9854-4702-b770-7e94f2497f57", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
106822025-04-30T06:44:07.803ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
106832025-04-30T06:44:07.806ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a69561ec-cc3a-461d-b1cb-65de06025b0a", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
106842025-04-30T06:44:07.806ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
106852025-04-30T06:44:07.808ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ff4efe97-f5e9-479a-a668-433dc46d86b8", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
106862025-04-30T06:44:07.808ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
106872025-04-30T06:44:07.809ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "276606d7-dccf-4545-b156-005be9905000", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
106882025-04-30T06:44:07.809ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
106892025-04-30T06:44:07.810ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a746f759-c3f0-4939-b5ab-c6ecde7e3b20", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
106902025-04-30T06:44:07.810ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
106912025-04-30T06:44:07.811ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c45934aa-06b7-4d52-b20b-8f14ff3a4166", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
106922025-04-30T06:44:07.811ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
106932025-04-30T06:44:07.812ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cb0b187f-39e8-486c-8a63-0092aa0ddf65", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
106942025-04-30T06:44:07.812ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
106952025-04-30T06:44:07.814ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0fe0e5ed-9a83-4c7a-8671-2fc54b7af1b9", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
106962025-04-30T06:44:07.814ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
106972025-04-30T06:44:07.816ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c4058939-8bcb-4798-863d-931859b3a07e", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
106982025-04-30T06:44:07.816ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
106992025-04-30T06:44:07.820ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ec78cf4e-f1c1-4983-ac89-b812ccf563d9", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
107002025-04-30T06:44:07.820ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
107012025-04-30T06:44:07.823ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
107022025-04-30T06:44:07.823ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
107032025-04-30T06:44:07.823ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
107042025-04-30T06:44:07.823ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
107052025-04-30T06:44:07.824ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
107062025-04-30T06:44:07.825ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5a6cca2f-49df-4ea8-90c5-e3103bab3f75", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
107072025-04-30T06:44:07.825ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
107082025-04-30T06:44:07.830ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "90849850-e9de-4e8b-83f2-84c2b548deda", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
107092025-04-30T06:44:07.830ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
107102025-04-30T06:44:07.830ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107112025-04-30T06:44:07.840ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
107122025-04-30T06:44:07.840ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
107132025-04-30T06:44:07.840ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
107142025-04-30T06:44:07.842ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
107152025-04-30T06:44:07.845ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
107162025-04-30T06:44:07.846ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
107172025-04-30T06:44:07.846ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
107182025-04-30T06:44:07.849ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "34563224-3200-479b-aa14-fdf3338a9880", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
107192025-04-30T06:44:07.849ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
107202025-04-30T06:44:07.859ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0cabb52-fcf1-4ae8-9ec5-3a931a9d26ef", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
107212025-04-30T06:44:07.859ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
107222025-04-30T06:44:07.861ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d4f76913-e4b6-45d0-a74d-78eb93fab155", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
107232025-04-30T06:44:07.862ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
107242025-04-30T06:44:07.866ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cdc5e331-7997-419e-8e22-161a477645e2", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
107252025-04-30T06:44:07.866ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
107262025-04-30T06:44:07.868ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "902b4427-060e-466e-a95d-2fc6adb791ff", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
107272025-04-30T06:44:07.868ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
107282025-04-30T06:44:07.870ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "384699e6-e3fd-4ff5-8407-9c9dad58982d", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
107292025-04-30T06:44:07.871ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
107302025-04-30T06:44:07.871ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
107312025-04-30T06:44:07.871ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
107322025-04-30T06:44:07.871ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
107332025-04-30T06:44:07.874ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
107342025-04-30T06:44:07.877ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107352025-04-30T06:44:07.885ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
107362025-04-30T06:44:07.885ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
107372025-04-30T06:44:07.885ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
107382025-04-30T06:44:07.885ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
107392025-04-30T06:44:07.885ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
107402025-04-30T06:44:07.886ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
107412025-04-30T06:44:07.891ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107422025-04-30T06:44:07.899ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
107432025-04-30T06:44:07.899ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
107442025-04-30T06:44:07.899ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
107452025-04-30T06:44:07.900ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
107462025-04-30T06:44:07.965ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
107472025-04-30T06:44:07.965ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory
107482025-04-30T06:44:07.966ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:34380 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
107492025-04-30T06:44:07.970ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:34380 req_id = a1a7c27f-35db-4aa3-99e9-f42d29d0494e sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
107502025-04-30T06:44:07.970ZINFOtest_unauthorized (dropshot): request completed latency_us = 600 local_addr = [::1]:57786 method = GET remote_addr = [::1]:34380 req_id = a1a7c27f-35db-4aa3-99e9-f42d29d0494e response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
107512025-04-30T06:44:07.971ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = http://[::1]:57786 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:57786/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "a1a7c27f-35db-4aa3-99e9-f42d29d0494e", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
107522025-04-30T06:44:07.972ZDEBGtest_unauthorized (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
107532025-04-30T06:44:07.972ZDEBGtest_unauthorized (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
107542025-04-30T06:44:07.972ZDEBGtest_unauthorized (BackgroundTasks): finished collection background_task = inventory_collection
107552025-04-30T06:44:07.974ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
107562025-04-30T06:44:07.974ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
107572025-04-30T06:44:07.975ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
107582025-04-30T06:44:07.975ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
107592025-04-30T06:44:07.976ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
107602025-04-30T06:44:07.977ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
107612025-04-30T06:44:07.980ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107622025-04-30T06:44:07.990ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
107632025-04-30T06:44:07.990ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
107642025-04-30T06:44:07.990ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
107652025-04-30T06:44:07.991ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
107662025-04-30T06:44:08.030ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
107672025-04-30T06:44:08.030ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
107682025-04-30T06:44:08.030ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
107692025-04-30T06:44:08.030ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
107702025-04-30T06:44:08.032ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
107712025-04-30T06:44:08.035ZWARNtest_unauthorized (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
107722025-04-30T06:44:08.035ZWARNtest_unauthorized (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
107732025-04-30T06:44:08.035ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 407.034369ms iteration = 37 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
107742025-04-30T06:44:08.035ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 38 reason = Signaled
107752025-04-30T06:44:08.035ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
107762025-04-30T06:44:08.035ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
107772025-04-30T06:44:08.035ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
107782025-04-30T06:44:08.035ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
107792025-04-30T06:44:08.035ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
107802025-04-30T06:44:08.036ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107812025-04-30T06:44:08.042ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
107822025-04-30T06:44:08.042ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
107832025-04-30T06:44:08.042ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
107842025-04-30T06:44:08.042ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
107852025-04-30T06:44:08.045ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107862025-04-30T06:44:08.052ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
107872025-04-30T06:44:08.052ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
107882025-04-30T06:44:08.052ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
107892025-04-30T06:44:08.053ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
107902025-04-30T06:44:08.067ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
107912025-04-30T06:44:08.067ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
107922025-04-30T06:44:08.068ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
107932025-04-30T06:44:08.070ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1a8b2318-abf8-406d-a389-112ab91a0de8, 0)]
107942025-04-30T06:44:08.070ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
107952025-04-30T06:44:08.074ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: a8 24 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .$..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:53327 req_id = 8837a84f-7d44-478e-826c-ce0a68c3d83f
107962025-04-30T06:44:08.074ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 43044,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:53327 req_id = 8837a84f-7d44-478e-826c-ce0a68c3d83f
107972025-04-30T06:44:08.074ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
107982025-04-30T06:44:08.074ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
107992025-04-30T06:44:08.074ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:53327 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 8837a84f-7d44-478e-826c-ce0a68c3d83f
108002025-04-30T06:44:08.076ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4312, tv_nsec: 875059728 } })
108012025-04-30T06:44:08.076ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
108022025-04-30T06:44:08.077ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 05 34 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .4..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:65236 req_id = fab1bb99-b0f6-49e9-a357-9d2d8a1d8361
108032025-04-30T06:44:08.078ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 1332,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:65236 req_id = fab1bb99-b0f6-49e9-a357-9d2d8a1d8361
108042025-04-30T06:44:08.078ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
108052025-04-30T06:44:08.078ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:65236 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = fab1bb99-b0f6-49e9-a357-9d2d8a1d8361
108062025-04-30T06:44:08.134ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
108072025-04-30T06:44:08.142ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 1e 40 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c .@..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:63290 req_id = 8f6b88c9-9c76-4e0c-be69-c55e6fba36cb
108082025-04-30T06:44:08.142ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 7744,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63290 req_id = 8f6b88c9-9c76-4e0c-be69-c55e6fba36cb
108092025-04-30T06:44:08.142ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
108102025-04-30T06:44:08.142ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 7744, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:63290 req_id = 8f6b88c9-9c76-4e0c-be69-c55e6fba36cb
108112025-04-30T06:44:08.143ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
108122025-04-30T06:44:08.143ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
108132025-04-30T06:44:08.143ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
108142025-04-30T06:44:08.147ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0e32b05-a8ae-4225-aa23-662a8e70bba4", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
108152025-04-30T06:44:08.148ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
108162025-04-30T06:44:08.152ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "db595a37-0bba-4440-adf3-9a4c37dc3a83", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
108172025-04-30T06:44:08.153ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
108182025-04-30T06:44:08.161ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "843ac771-b5e8-4f3f-a424-66ae0358f0b9", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
108192025-04-30T06:44:08.162ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
108202025-04-30T06:44:08.167ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "359bb1d0-e1d3-41bf-9af2-019924b318d3", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
108212025-04-30T06:44:08.168ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
108222025-04-30T06:44:08.173ZDEBGtest_unauthorized (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
108232025-04-30T06:44:08.174ZDEBGtest_unauthorized (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
108242025-04-30T06:44:08.174ZDEBGtest_unauthorized (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a
108252025-04-30T06:44:08.176ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
108262025-04-30T06:44:08.183ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
108272025-04-30T06:44:08.187ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 2 table_name = fields_string
108282025-04-30T06:44:08.187ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
108292025-04-30T06:44:08.190ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "40340c59-b9ed-488f-bbde-46615e2aaf8c", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
108302025-04-30T06:44:08.190ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
108312025-04-30T06:44:08.192ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 1 table_name = measurements_i64
108322025-04-30T06:44:08.192ZTRACtest_unauthorized (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
108332025-04-30T06:44:08.195ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8d1b0413-83ca-4762-a8b1-c8bbf80569bb", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
108342025-04-30T06:44:08.195ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
108352025-04-30T06:44:08.198ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "01050350-a9b0-443f-9ccf-8328a0c7635d", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
108362025-04-30T06:44:08.199ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
108372025-04-30T06:44:08.200ZDEBGtest_unauthorized (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
108382025-04-30T06:44:08.200ZDEBGtest_unauthorized (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
108392025-04-30T06:44:08.202ZTRACtest_unauthorized (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
108402025-04-30T06:44:08.203ZDEBGtest_unauthorized (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
108412025-04-30T06:44:08.203ZTRACtest_unauthorized (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
108422025-04-30T06:44:08.203ZTRACtest_unauthorized (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
108432025-04-30T06:44:08.205ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e341d2b4-fd96-426f-8744-ac153931dcf1", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
108442025-04-30T06:44:08.205ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
108452025-04-30T06:44:08.207ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "eccc2a91-0a00-4701-9d62-2b503090a808", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
108462025-04-30T06:44:08.208ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
108472025-04-30T06:44:08.209ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "906200d7-b436-40b0-a562-ed5017b1b6cc", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
108482025-04-30T06:44:08.210ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
108492025-04-30T06:44:08.212ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "777e18a0-c263-4241-a71c-adec675765e3", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
108502025-04-30T06:44:08.212ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
108512025-04-30T06:44:08.213ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7d114c59-a366-4591-b0a1-fed1f46bffc9", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
108522025-04-30T06:44:08.213ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
108532025-04-30T06:44:08.215ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8ba6eb9b-295a-4859-92d9-eaaf3399898f", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
108542025-04-30T06:44:08.215ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
108552025-04-30T06:44:08.218ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e602d618-28ba-4410-8f04-73e986a46f5a", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
108562025-04-30T06:44:08.218ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
108572025-04-30T06:44:08.219ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c8444b9e-35c3-4578-86e6-34b8d35b35da", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
108582025-04-30T06:44:08.219ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
108592025-04-30T06:44:08.221ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "80cd8143-4336-4eb8-a85c-ba078024a9c7", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
108602025-04-30T06:44:08.222ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
108612025-04-30T06:44:08.223ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e7e60da1-4e27-408f-9d0d-242cdf4664b3", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
108622025-04-30T06:44:08.223ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
108632025-04-30T06:44:08.230ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b0ed9f12-279f-4eec-89fa-05a25d6b8332", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
108642025-04-30T06:44:08.230ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
108652025-04-30T06:44:08.233ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7ef77e76-2086-442a-9dfb-c894908c91fc", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
108662025-04-30T06:44:08.234ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
108672025-04-30T06:44:08.234ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
108682025-04-30T06:44:08.234ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
108692025-04-30T06:44:08.235ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
108702025-04-30T06:44:08.235ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
108712025-04-30T06:44:08.236ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
108722025-04-30T06:44:08.237ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "777577b8-7b9d-4c5b-a0bf-d2450c719cc9", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
108732025-04-30T06:44:08.237ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
108742025-04-30T06:44:08.239ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c53bf89e-679f-434e-b16b-3c5ec1d31c5c", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
108752025-04-30T06:44:08.239ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
108762025-04-30T06:44:08.240ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108772025-04-30T06:44:08.246ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
108782025-04-30T06:44:08.246ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
108792025-04-30T06:44:08.246ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
108802025-04-30T06:44:08.247ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
108812025-04-30T06:44:08.252ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
108822025-04-30T06:44:08.252ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
108832025-04-30T06:44:08.255ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
108842025-04-30T06:44:08.256ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "db0fe73c-f393-4f39-bba3-d832f23992d2", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
108852025-04-30T06:44:08.256ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
108862025-04-30T06:44:08.260ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f92bb22b-7bfd-4ffa-b907-1d573dcdd642", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
108872025-04-30T06:44:08.260ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
108882025-04-30T06:44:08.264ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a29f3386-78f7-40ac-b7fa-b170736d787b", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
108892025-04-30T06:44:08.265ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
108902025-04-30T06:44:08.273ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0f902b9f-2aeb-49f8-8ef3-1471f69d45ec", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
108912025-04-30T06:44:08.273ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
108922025-04-30T06:44:08.275ZWARNtest_unauthorized (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
108932025-04-30T06:44:08.280ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7681dcdb-400b-4a39-a566-64cfaa4af894", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
108942025-04-30T06:44:08.280ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
108952025-04-30T06:44:08.297ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "58e2c306-0f9d-4b3a-9510-c57792549215", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
108962025-04-30T06:44:08.297ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
108972025-04-30T06:44:08.298ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
108982025-04-30T06:44:08.302ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "91f29883-7789-4e46-9d38-255461bffde2", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
108992025-04-30T06:44:08.302ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
109002025-04-30T06:44:08.306ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b6d927de-1404-44bc-be6d-985760ea89c5", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
109012025-04-30T06:44:08.307ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
109022025-04-30T06:44:08.311ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e998c089-487f-4b77-ad05-448feafe3897", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
109032025-04-30T06:44:08.312ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
109042025-04-30T06:44:08.314ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c50c3393-24d4-41ec-8a8d-4c118f3ff444", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
109052025-04-30T06:44:08.314ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
109062025-04-30T06:44:08.317ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d597e0d5-674d-4df1-8005-d66215850611", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
109072025-04-30T06:44:08.317ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
109082025-04-30T06:44:08.329ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d3d574fb-8e79-4533-b80d-c7e3b15afdf6", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
109092025-04-30T06:44:08.329ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
109102025-04-30T06:44:08.330ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "74d5b313-34fb-488a-a8e9-5ea1f71efa94", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
109112025-04-30T06:44:08.330ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
109122025-04-30T06:44:08.332ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2bb51695-2ac6-4518-8893-803c673693c9", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
109132025-04-30T06:44:08.333ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
109142025-04-30T06:44:08.339ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c7acf9db-67ea-42cd-a109-ab5bf2e973e0", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
109152025-04-30T06:44:08.339ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
109162025-04-30T06:44:08.344ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "14f79a77-c673-4cc2-a697-62ee44fdec63", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
109172025-04-30T06:44:08.344ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
109182025-04-30T06:44:08.349ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
109192025-04-30T06:44:08.349ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
109202025-04-30T06:44:08.349ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
109212025-04-30T06:44:08.349ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
109222025-04-30T06:44:08.350ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
109232025-04-30T06:44:08.350ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b670a692-2d71-442d-992e-e0f030cea325", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
109242025-04-30T06:44:08.350ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
109252025-04-30T06:44:08.355ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109262025-04-30T06:44:08.362ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
109272025-04-30T06:44:08.362ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
109282025-04-30T06:44:08.362ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
109292025-04-30T06:44:08.363ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
109302025-04-30T06:44:08.366ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "47e0a46e-67c8-4652-8ab4-d5ca6f42841e", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
109312025-04-30T06:44:08.366ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
109322025-04-30T06:44:08.376ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
109332025-04-30T06:44:08.376ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
109342025-04-30T06:44:08.376ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
109352025-04-30T06:44:08.379ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a20ac90f-eedd-4434-bc2f-88786090a735", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
109362025-04-30T06:44:08.379ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
109372025-04-30T06:44:08.384ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "eaf47e3f-9da5-4cfe-ac8c-ee192da112c6", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
109382025-04-30T06:44:08.384ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
109392025-04-30T06:44:08.385ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "36f4217b-0c15-43f2-97f4-b1cb75846e62", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
109402025-04-30T06:44:08.386ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
109412025-04-30T06:44:08.387ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "17d4e99d-3580-44ca-b33c-1ea27ecaad0d", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
109422025-04-30T06:44:08.388ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
109432025-04-30T06:44:08.397ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b22f9d9a-4fa8-43ef-8ae0-91d208134ca7", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
109442025-04-30T06:44:08.399ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
109452025-04-30T06:44:08.402ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "599ba684-77da-4fbd-9825-bf2f78d32a49", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
109462025-04-30T06:44:08.402ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
109472025-04-30T06:44:08.402ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
109482025-04-30T06:44:08.402ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
109492025-04-30T06:44:08.402ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
109502025-04-30T06:44:08.403ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
109512025-04-30T06:44:08.405ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109522025-04-30T06:44:08.416ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
109532025-04-30T06:44:08.416ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
109542025-04-30T06:44:08.416ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
109552025-04-30T06:44:08.416ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
109562025-04-30T06:44:08.416ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
109572025-04-30T06:44:08.419ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
109582025-04-30T06:44:08.422ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109592025-04-30T06:44:08.436ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
109602025-04-30T06:44:08.436ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
109612025-04-30T06:44:08.436ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
109622025-04-30T06:44:08.436ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
109632025-04-30T06:44:08.514ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
109642025-04-30T06:44:08.514ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory
109652025-04-30T06:44:08.514ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:44215 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
109662025-04-30T06:44:08.515ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:44215 req_id = 34b1ffd4-8b43-4a36-a84a-16fceca1ca19 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
109672025-04-30T06:44:08.515ZINFOtest_unauthorized (dropshot): request completed latency_us = 586 local_addr = [::1]:57786 method = GET remote_addr = [::1]:44215 req_id = 34b1ffd4-8b43-4a36-a84a-16fceca1ca19 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
109682025-04-30T06:44:08.515ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = http://[::1]:57786 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:57786/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "34b1ffd4-8b43-4a36-a84a-16fceca1ca19", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
109692025-04-30T06:44:08.518ZDEBGtest_unauthorized (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
109702025-04-30T06:44:08.518ZDEBGtest_unauthorized (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
109712025-04-30T06:44:08.518ZDEBGtest_unauthorized (BackgroundTasks): finished collection background_task = inventory_collection
109722025-04-30T06:44:08.520ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
109732025-04-30T06:44:08.520ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
109742025-04-30T06:44:08.520ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
109752025-04-30T06:44:08.520ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
109762025-04-30T06:44:08.520ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
109772025-04-30T06:44:08.524ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109782025-04-30T06:44:08.528ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
109792025-04-30T06:44:08.528ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
109802025-04-30T06:44:08.528ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
109812025-04-30T06:44:08.529ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
109822025-04-30T06:44:08.535ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
109832025-04-30T06:44:08.586ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
109842025-04-30T06:44:08.586ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
109852025-04-30T06:44:08.586ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
109862025-04-30T06:44:08.586ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
109872025-04-30T06:44:08.586ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
109882025-04-30T06:44:08.594ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109892025-04-30T06:44:08.599ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
109902025-04-30T06:44:08.599ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
109912025-04-30T06:44:08.599ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
109922025-04-30T06:44:08.600ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
109932025-04-30T06:44:08.613ZTRACtest_unauthorized (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
109942025-04-30T06:44:08.623ZWARNtest_unauthorized (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
109952025-04-30T06:44:08.624ZWARNtest_unauthorized (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
109962025-04-30T06:44:08.624ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 583.621879ms iteration = 38 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
109972025-04-30T06:44:08.624ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 39 reason = Signaled
109982025-04-30T06:44:08.624ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
109992025-04-30T06:44:08.624ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
110002025-04-30T06:44:08.624ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
110012025-04-30T06:44:08.624ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
110022025-04-30T06:44:08.626ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
110032025-04-30T06:44:08.627ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
110042025-04-30T06:44:08.627ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
110052025-04-30T06:44:08.628ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
110062025-04-30T06:44:08.628ZTRACtest_unauthorized (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: a6 43 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 .C..........._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:55792 req_id = cc0c3b38-8f5c-4bbd-a1fa-4d13c1c77920
110072025-04-30T06:44:08.628ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 42563,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:55792 req_id = cc0c3b38-8f5c-4bbd-a1fa-4d13c1c77920
110082025-04-30T06:44:08.628ZDEBGtest_unauthorized (store): query key key = _dendrite._tcp
110092025-04-30T06:44:08.628ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
110102025-04-30T06:44:08.629ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:55792 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 44264, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = cc0c3b38-8f5c-4bbd-a1fa-4d13c1c77920
110112025-04-30T06:44:08.630ZDEBGtest_unauthorized (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 44264, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4313, tv_nsec: 428997443 } })
110122025-04-30T06:44:08.630ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
110132025-04-30T06:44:08.631ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: fb 10 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:65068 req_id = 679bfc74-b874-457b-b2bb-22a4502bd24e
110142025-04-30T06:44:08.631ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 64272,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:65068 req_id = 679bfc74-b874-457b-b2bb-22a4502bd24e
110152025-04-30T06:44:08.634ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
110162025-04-30T06:44:08.634ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:65068 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 679bfc74-b874-457b-b2bb-22a4502bd24e
110172025-04-30T06:44:08.634ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110182025-04-30T06:44:08.646ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
110192025-04-30T06:44:08.646ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
110202025-04-30T06:44:08.646ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
110212025-04-30T06:44:08.647ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
110222025-04-30T06:44:08.648ZINFOtest_unauthorized (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector
110232025-04-30T06:44:08.765ZINFOtest_unauthorized (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector zone_address = ::1
110242025-04-30T06:44:08.765ZDEBGtest_unauthorized (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[::1]:12225/local/switch-id
110252025-04-30T06:44:08.766ZDEBGtest_unauthorized (MgsClient): client response background_task = nat_v4_garbage_collector result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
110262025-04-30T06:44:08.766ZWARNtest_unauthorized (BackgroundTasks): failed to identify switch slot for dendrite background_task = nat_v4_garbage_collector reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
110272025-04-30T06:44:08.768ZWARNtest_unauthorized (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = nat_v4_garbage_collector
110282025-04-30T06:44:08.769ZTRACtest_unauthorized (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
110292025-04-30T06:44:08.772ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1a8b2318-abf8-406d-a389-112ab91a0de8, 0)]
110302025-04-30T06:44:08.772ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
110312025-04-30T06:44:08.773ZTRACtest_unauthorized (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 8f a7 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 ............._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:54669 req_id = efa7ca45-62fa-4c69-a082-ec43d85501e9
110322025-04-30T06:44:08.773ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 36775,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54669 req_id = efa7ca45-62fa-4c69-a082-ec43d85501e9
110332025-04-30T06:44:08.773ZDEBGtest_unauthorized (store): query key key = _dendrite._tcp
110342025-04-30T06:44:08.773ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
110352025-04-30T06:44:08.774ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:54669 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 44264, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = efa7ca45-62fa-4c69-a082-ec43d85501e9
110362025-04-30T06:44:08.774ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 92 ad 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:58389 req_id = 11536384-d888-4bac-9f71-34cfc5dd015d
110372025-04-30T06:44:08.774ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 37549,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:58389 req_id = 11536384-d888-4bac-9f71-34cfc5dd015d
110382025-04-30T06:44:08.774ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
110392025-04-30T06:44:08.775ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
110402025-04-30T06:44:08.775ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:58389 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 11536384-d888-4bac-9f71-34cfc5dd015d
110412025-04-30T06:44:08.776ZDEBGtest_unauthorized (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 44264, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4313, tv_nsec: 575370913 } })
110422025-04-30T06:44:08.776ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
110432025-04-30T06:44:08.776ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4313, tv_nsec: 575715981 } })
110442025-04-30T06:44:08.776ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
110452025-04-30T06:44:08.777ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: bf b3 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:49853 req_id = d54a8f39-fd9f-4e33-a41e-873e4eca9f54
110462025-04-30T06:44:08.777ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 49075,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:49853 req_id = d54a8f39-fd9f-4e33-a41e-873e4eca9f54
110472025-04-30T06:44:08.777ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
110482025-04-30T06:44:08.777ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:49853 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = d54a8f39-fd9f-4e33-a41e-873e4eca9f54
110492025-04-30T06:44:08.777ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: f3 94 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:56604 req_id = bb948fc4-4782-4359-aa3b-e432d7279a25
110502025-04-30T06:44:08.777ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 62356,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:56604 req_id = bb948fc4-4782-4359-aa3b-e432d7279a25
110512025-04-30T06:44:08.781ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
110522025-04-30T06:44:08.781ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:56604 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = bb948fc4-4782-4359-aa3b-e432d7279a25
110532025-04-30T06:44:08.783ZINFOtest_unauthorized (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager
110542025-04-30T06:44:08.851ZINFOtest_unauthorized (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager zone_address = ::1
110552025-04-30T06:44:08.851ZDEBGtest_unauthorized (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[::1]:12225/local/switch-id
110562025-04-30T06:44:08.852ZDEBGtest_unauthorized (MgsClient): client response background_task = bfd_manager result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
110572025-04-30T06:44:08.852ZWARNtest_unauthorized (BackgroundTasks): failed to identify switch slot for dendrite background_task = bfd_manager reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
110582025-04-30T06:44:08.855ZWARNtest_unauthorized (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = bfd_manager
110592025-04-30T06:44:08.924ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
110602025-04-30T06:44:08.937ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 36 0b 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c 6............_cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:52673 req_id = 99cd49b4-3e08-4add-a638-65084915a85d
110612025-04-30T06:44:08.937ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 13835,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52673 req_id = 99cd49b4-3e08-4add-a638-65084915a85d
110622025-04-30T06:44:08.939ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
110632025-04-30T06:44:08.939ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 13835, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:52673 req_id = 99cd49b4-3e08-4add-a638-65084915a85d
110642025-04-30T06:44:08.940ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
110652025-04-30T06:44:08.940ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
110662025-04-30T06:44:08.940ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
110672025-04-30T06:44:08.947ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "05ab1f2c-a831-4969-9f5b-68748aba6a06", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
110682025-04-30T06:44:08.947ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
110692025-04-30T06:44:08.950ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "48981820-c678-43b6-bffd-ed6630f47950", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
110702025-04-30T06:44:08.950ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
110712025-04-30T06:44:08.957ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "71fcaac6-67be-45d4-a1a5-4bfa1954aa31", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
110722025-04-30T06:44:08.958ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
110732025-04-30T06:44:08.966ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "29eb07ea-338e-4b63-a4e9-983aba1b501b", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
110742025-04-30T06:44:08.966ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
110752025-04-30T06:44:08.968ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
110762025-04-30T06:44:08.976ZTRACtest_unauthorized (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
110772025-04-30T06:44:08.977ZTRACtest_unauthorized (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: e1 53 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 .S..........._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:59210 req_id = 072d6e03-8709-4c67-8c83-b39aa216edc8
110782025-04-30T06:44:08.977ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 57683,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:59210 req_id = 072d6e03-8709-4c67-8c83-b39aa216edc8
110792025-04-30T06:44:08.978ZDEBGtest_unauthorized (store): query key key = _dendrite._tcp
110802025-04-30T06:44:08.978ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
110812025-04-30T06:44:08.978ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:59210 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 44264, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 072d6e03-8709-4c67-8c83-b39aa216edc8
110822025-04-30T06:44:08.979ZDEBGtest_unauthorized (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 44264, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4313, tv_nsec: 778715143 } })
110832025-04-30T06:44:08.979ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
110842025-04-30T06:44:08.980ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 65 75 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e eu..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:58685 req_id = bd9e38a5-a3a1-47df-99c2-1cce3c4ee32a
110852025-04-30T06:44:08.980ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 25973,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:58685 req_id = bd9e38a5-a3a1-47df-99c2-1cce3c4ee32a
110862025-04-30T06:44:08.981ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
110872025-04-30T06:44:08.981ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:58685 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = bd9e38a5-a3a1-47df-99c2-1cce3c4ee32a
110882025-04-30T06:44:08.982ZINFOtest_unauthorized (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc
110892025-04-30T06:44:09.042ZINFOtest_unauthorized (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc zone_address = ::1
110902025-04-30T06:44:09.042ZDEBGtest_unauthorized (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc uri = http://[::1]:12225/local/switch-id
110912025-04-30T06:44:09.043ZDEBGtest_unauthorized (MgsClient): client response background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
110922025-04-30T06:44:09.043ZWARNtest_unauthorized (BackgroundTasks): failed to identify switch slot for dendrite background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
110932025-04-30T06:44:09.044ZWARNtest_unauthorized (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc
110942025-04-30T06:44:09.044ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
110952025-04-30T06:44:09.044ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
110962025-04-30T06:44:09.044ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
110972025-04-30T06:44:09.044ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
110982025-04-30T06:44:09.045ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
110992025-04-30T06:44:09.050ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c98ffe75-11e0-4cf9-bdb5-e08ae3312d58", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
111002025-04-30T06:44:09.051ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
111012025-04-30T06:44:09.051ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111022025-04-30T06:44:09.057ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
111032025-04-30T06:44:09.057ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
111042025-04-30T06:44:09.057ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
111052025-04-30T06:44:09.058ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
111062025-04-30T06:44:09.061ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
111072025-04-30T06:44:09.061ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
111082025-04-30T06:44:09.062ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
111092025-04-30T06:44:09.063ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7a24ee58-525a-404b-b02d-55af9612929e", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
111102025-04-30T06:44:09.064ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
111112025-04-30T06:44:09.071ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "360859be-aad4-460a-8d4c-f298a9915c83", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
111122025-04-30T06:44:09.071ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
111132025-04-30T06:44:09.077ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d1628ad1-9dcb-4287-b711-4154382124a6", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
111142025-04-30T06:44:09.077ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
111152025-04-30T06:44:09.079ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "626b36b4-720c-4bdf-ba78-88101e8d5ea4", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
111162025-04-30T06:44:09.079ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
111172025-04-30T06:44:09.080ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "af5293c6-77d0-4cb4-9e63-5a7b13a583ca", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
111182025-04-30T06:44:09.081ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
111192025-04-30T06:44:09.088ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "19b2fe74-fe5f-4fef-b875-480021f7472d", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
111202025-04-30T06:44:09.089ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
111212025-04-30T06:44:09.094ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "440255c5-d910-432d-b645-1d59a27f5c5d", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
111222025-04-30T06:44:09.094ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
111232025-04-30T06:44:09.098ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ba6931dc-1cb0-4c62-81bd-1cf55395421f", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
111242025-04-30T06:44:09.098ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
111252025-04-30T06:44:09.099ZINFOtest_unauthorized (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
111262025-04-30T06:44:09.099ZDEBGtest_unauthorized (oximeter): client request body = None method = GET uri = http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
111272025-04-30T06:44:09.105ZTRACtest_unauthorized (dropshot_internal): incoming request local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 462ef4d0-940a-41e3-a569-f9b0249e0b9e uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
111282025-04-30T06:44:09.106ZTRACtest_unauthorized (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 462ef4d0-940a-41e3-a569-f9b0249e0b9e resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
111292025-04-30T06:44:09.106ZDEBGtest_unauthorized (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 462ef4d0-940a-41e3-a569-f9b0249e0b9e roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
111302025-04-30T06:44:09.107ZDEBGtest_unauthorized (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 462ef4d0-940a-41e3-a569-f9b0249e0b9e resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
111312025-04-30T06:44:09.107ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
111322025-04-30T06:44:09.110ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "feba3e6f-b08a-4406-9986-afa5785438ec", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
111332025-04-30T06:44:09.110ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
111342025-04-30T06:44:09.111ZINFOtest_unauthorized (dropshot_internal): request completed latency_us = 5038 local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 462ef4d0-940a-41e3-a569-f9b0249e0b9e response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
111352025-04-30T06:44:09.111ZDEBGtest_unauthorized (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "462ef4d0-940a-41e3-a569-f9b0249e0b9e", "content-length": "405", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
111362025-04-30T06:44:09.112ZDEBGtest_unauthorized (oximeter): client request body = None method = GET uri = http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
111372025-04-30T06:44:09.112ZTRACtest_unauthorized (dropshot_internal): incoming request local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = a5079bb1-402e-45d1-9de2-19373169d156 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
111382025-04-30T06:44:09.112ZTRACtest_unauthorized (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = a5079bb1-402e-45d1-9de2-19373169d156 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
111392025-04-30T06:44:09.113ZDEBGtest_unauthorized (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = a5079bb1-402e-45d1-9de2-19373169d156 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
111402025-04-30T06:44:09.113ZDEBGtest_unauthorized (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = a5079bb1-402e-45d1-9de2-19373169d156 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
111412025-04-30T06:44:09.116ZINFOtest_unauthorized (dropshot_internal): request completed latency_us = 3889 local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = a5079bb1-402e-45d1-9de2-19373169d156 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
111422025-04-30T06:44:09.116ZDEBGtest_unauthorized (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 8d752adb-08b9-405f-a11b-66019bf447b5
111432025-04-30T06:44:09.117ZDEBGtest_unauthorized (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "a5079bb1-402e-45d1-9de2-19373169d156", "content-length": "29", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
111442025-04-30T06:44:09.117ZTRACtest_unauthorized (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 8d752adb-08b9-405f-a11b-66019bf447b5
111452025-04-30T06:44:09.117ZTRACtest_unauthorized (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
111462025-04-30T06:44:09.117ZINFOtest_unauthorized (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
111472025-04-30T06:44:09.117ZDEBGtest_unauthorized (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 8d752adb-08b9-405f-a11b-66019bf447b5
111482025-04-30T06:44:09.118ZTRACtest_unauthorized (dropshot): incoming request local_addr = 127.0.0.1:53884 method = GET remote_addr = 127.0.0.1:47166 req_id = 5c0d5e58-aeb2-44f7-b593-3abdde8c8899 uri = /8d752adb-08b9-405f-a11b-66019bf447b5
111492025-04-30T06:44:09.125ZINFOtest_unauthorized (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 7310 local_addr = 127.0.0.1:53884 method = GET remote_addr = 127.0.0.1:47166 req_id = 5c0d5e58-aeb2-44f7-b593-3abdde8c8899 response_code = 200 uri = /8d752adb-08b9-405f-a11b-66019bf447b5
111502025-04-30T06:44:09.131ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4e127c0f-e1d3-4a04-8fd3-ae42ba3acfc1", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
111512025-04-30T06:44:09.132ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
111522025-04-30T06:44:09.132ZTRACtest_unauthorized (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 8d752adb-08b9-405f-a11b-66019bf447b5
111532025-04-30T06:44:09.138ZDEBGtest_unauthorized (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 5 producer_id = 8d752adb-08b9-405f-a11b-66019bf447b5
111542025-04-30T06:44:09.138ZTRACtest_unauthorized (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 8d752adb-08b9-405f-a11b-66019bf447b5
111552025-04-30T06:44:09.138ZTRACtest_unauthorized (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 8d752adb-08b9-405f-a11b-66019bf447b5
111562025-04-30T06:44:09.157ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6d5625dc-d056-452b-99af-362f4f932c07", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
111572025-04-30T06:44:09.158ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
111582025-04-30T06:44:09.159ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
111592025-04-30T06:44:09.160ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
111602025-04-30T06:44:09.160ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
111612025-04-30T06:44:09.160ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
111622025-04-30T06:44:09.160ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
111632025-04-30T06:44:09.163ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "df6005e8-01b6-4820-b710-3acb014851e2", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
111642025-04-30T06:44:09.164ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
111652025-04-30T06:44:09.164ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111662025-04-30T06:44:09.172ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
111672025-04-30T06:44:09.172ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
111682025-04-30T06:44:09.172ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
111692025-04-30T06:44:09.172ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
111702025-04-30T06:44:09.174ZDEBGtest_unauthorized (results-sink): inserting 6 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
111712025-04-30T06:44:09.174ZDEBGtest_unauthorized (results-sink-cluster): inserting 6 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
111722025-04-30T06:44:09.174ZDEBGtest_unauthorized (clickhouse-client): unrolling 6 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a
111732025-04-30T06:44:09.181ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 12 sql = INSERT INTO oximeter.fields_string FORMAT Native
111742025-04-30T06:44:09.192ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
111752025-04-30T06:44:09.194ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
111762025-04-30T06:44:09.197ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
111772025-04-30T06:44:09.202ZDEBGtest_unauthorized (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
111782025-04-30T06:44:09.203ZDEBGtest_unauthorized (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
111792025-04-30T06:44:09.211ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 12 table_name = fields_string
111802025-04-30T06:44:09.211ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 5 sql = INSERT INTO oximeter.fields_u16 FORMAT Native
111812025-04-30T06:44:09.213ZTRACtest_unauthorized (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
111822025-04-30T06:44:09.213ZDEBGtest_unauthorized (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
111832025-04-30T06:44:09.213ZTRACtest_unauthorized (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
111842025-04-30T06:44:09.213ZTRACtest_unauthorized (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
111852025-04-30T06:44:09.216ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 5 table_name = fields_u16
111862025-04-30T06:44:09.216ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 5 sql = INSERT INTO oximeter.fields_uuid FORMAT Native
111872025-04-30T06:44:09.223ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 5 table_name = fields_uuid
111882025-04-30T06:44:09.223ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 19 n_rows = 5 sql = INSERT INTO oximeter.measurements_histogramu64 FORMAT Native
111892025-04-30T06:44:09.228ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "23557c0e-b006-487d-ac8b-e6b955d36bf6", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
111902025-04-30T06:44:09.229ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
111912025-04-30T06:44:09.232ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 5 table_name = measurements_histogramu64
111922025-04-30T06:44:09.234ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
111932025-04-30T06:44:09.237ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 1 table_name = measurements_i64
111942025-04-30T06:44:09.237ZTRACtest_unauthorized (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
111952025-04-30T06:44:09.244ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5ecd02d8-a256-409f-9cee-b6f3dcce2b62", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
111962025-04-30T06:44:09.244ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
111972025-04-30T06:44:09.245ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "171afbb4-f1f1-4d23-b43d-d06e1ef15b20", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
111982025-04-30T06:44:09.246ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
111992025-04-30T06:44:09.252ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a25656f0-5ef8-47e5-8520-fb8ae4ec7539", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
112002025-04-30T06:44:09.252ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
112012025-04-30T06:44:09.258ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
112022025-04-30T06:44:09.259ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3e1a7591-b825-46cc-b723-5afae1771149", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
112032025-04-30T06:44:09.259ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
112042025-04-30T06:44:09.272ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ba328344-b047-4b52-8be9-640048cb7153", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
112052025-04-30T06:44:09.272ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
112062025-04-30T06:44:09.277ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c718d3df-8e30-4c36-ab98-fc97b89d98d7", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
112072025-04-30T06:44:09.277ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
112082025-04-30T06:44:09.277ZWARNtest_unauthorized (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
112092025-04-30T06:44:09.283ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "072b206c-fd11-4f1f-9b21-f669bae4c9cd", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
112102025-04-30T06:44:09.283ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
112112025-04-30T06:44:09.287ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "95966881-8f94-4648-b60b-04b33851b906", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
112122025-04-30T06:44:09.287ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
112132025-04-30T06:44:09.291ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b7ca3335-e0b9-4dc5-a339-85949dfa96ae", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
112142025-04-30T06:44:09.291ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
112152025-04-30T06:44:09.293ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "dec2598b-1207-452d-9b67-bbde81fa257a", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
112162025-04-30T06:44:09.293ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
112172025-04-30T06:44:09.295ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "10047a2e-2ff1-4d17-9297-533c93b1406c", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
112182025-04-30T06:44:09.295ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
112192025-04-30T06:44:09.298ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "92dd1914-d3e2-40f0-ad8f-808564d386b5", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
112202025-04-30T06:44:09.298ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
112212025-04-30T06:44:09.299ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c06384d2-6b00-4a9f-9d2a-cf4794ebbef9", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
112222025-04-30T06:44:09.299ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
112232025-04-30T06:44:09.301ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "786c96f4-4a5a-4e5c-bd19-5e1a749d3bd8", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
112242025-04-30T06:44:09.301ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
112252025-04-30T06:44:09.302ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3805a26e-d67a-4427-a94a-0ea22cb82dd3", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
112262025-04-30T06:44:09.303ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
112272025-04-30T06:44:09.304ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ea321854-6796-450d-8ab7-2f9de281f19e", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
112282025-04-30T06:44:09.304ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
112292025-04-30T06:44:09.306ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "18c5bf2e-6914-4b12-8f76-3d6d4e79dec7", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
112302025-04-30T06:44:09.306ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
112312025-04-30T06:44:09.308ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "17de3e96-313c-436d-8200-952b131e2eb9", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
112322025-04-30T06:44:09.308ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
112332025-04-30T06:44:09.309ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
112342025-04-30T06:44:09.309ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
112352025-04-30T06:44:09.309ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
112362025-04-30T06:44:09.309ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
112372025-04-30T06:44:09.310ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
112382025-04-30T06:44:09.313ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112392025-04-30T06:44:09.319ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
112402025-04-30T06:44:09.319ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
112412025-04-30T06:44:09.319ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
112422025-04-30T06:44:09.320ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
112432025-04-30T06:44:09.324ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a43ec13f-72ab-4519-a2b5-f05a71ffa667", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
112442025-04-30T06:44:09.324ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
112452025-04-30T06:44:09.324ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
112462025-04-30T06:44:09.325ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
112472025-04-30T06:44:09.325ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
112482025-04-30T06:44:09.330ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "44837c69-f689-41fd-88a9-a0a8e6b47a64", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
112492025-04-30T06:44:09.330ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
112502025-04-30T06:44:09.335ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a0a2292-2795-4fbe-b71c-77c47cfbe6a0", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
112512025-04-30T06:44:09.335ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
112522025-04-30T06:44:09.344ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0d61e455-18cf-474e-b211-f3cea8857124", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
112532025-04-30T06:44:09.344ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
112542025-04-30T06:44:09.355ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "536379d3-4447-448b-b74b-ca7d64700b98", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
112552025-04-30T06:44:09.355ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
112562025-04-30T06:44:09.357ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd1e5d3f-01d1-4b75-ab7a-c7bb10cc539e", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
112572025-04-30T06:44:09.357ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
112582025-04-30T06:44:09.361ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
112592025-04-30T06:44:09.361ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1420dfeb-7a47-4359-aac6-3d380c3080a2", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
112602025-04-30T06:44:09.361ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
112612025-04-30T06:44:09.364ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "10ff15a5-8ce0-4bb0-adb0-f1042af288fb", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
112622025-04-30T06:44:09.364ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
112632025-04-30T06:44:09.366ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "18b48043-f713-494a-981e-8dde6c7a3d3e", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
112642025-04-30T06:44:09.366ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
112652025-04-30T06:44:09.366ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
112662025-04-30T06:44:09.366ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
112672025-04-30T06:44:09.366ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
112682025-04-30T06:44:09.367ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
112692025-04-30T06:44:09.372ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112702025-04-30T06:44:09.378ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
112712025-04-30T06:44:09.378ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
112722025-04-30T06:44:09.378ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
112732025-04-30T06:44:09.378ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
112742025-04-30T06:44:09.378ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
112752025-04-30T06:44:09.378ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
112762025-04-30T06:44:09.381ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112772025-04-30T06:44:09.389ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
112782025-04-30T06:44:09.389ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
112792025-04-30T06:44:09.389ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
112802025-04-30T06:44:09.391ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
112812025-04-30T06:44:09.509ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
112822025-04-30T06:44:09.510ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory
112832025-04-30T06:44:09.511ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
112842025-04-30T06:44:09.512ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
112852025-04-30T06:44:09.512ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
112862025-04-30T06:44:09.512ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
112872025-04-30T06:44:09.513ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
112882025-04-30T06:44:09.513ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:52810 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
112892025-04-30T06:44:09.515ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:52810 req_id = 5499d4ae-8cc6-4ebc-954c-2a64172230f7 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
112902025-04-30T06:44:09.515ZINFOtest_unauthorized (dropshot): request completed latency_us = 687 local_addr = [::1]:57786 method = GET remote_addr = [::1]:52810 req_id = 5499d4ae-8cc6-4ebc-954c-2a64172230f7 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
112912025-04-30T06:44:09.516ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = http://[::1]:57786 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:57786/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "5499d4ae-8cc6-4ebc-954c-2a64172230f7", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
112922025-04-30T06:44:09.519ZDEBGtest_unauthorized (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
112932025-04-30T06:44:09.519ZDEBGtest_unauthorized (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
112942025-04-30T06:44:09.519ZDEBGtest_unauthorized (BackgroundTasks): finished collection background_task = inventory_collection
112952025-04-30T06:44:09.522ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
112962025-04-30T06:44:09.522ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
112972025-04-30T06:44:09.522ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
112982025-04-30T06:44:09.522ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
112992025-04-30T06:44:09.523ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
113002025-04-30T06:44:09.524ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113012025-04-30T06:44:09.528ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
113022025-04-30T06:44:09.529ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
113032025-04-30T06:44:09.529ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
113042025-04-30T06:44:09.529ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
113052025-04-30T06:44:09.531ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113062025-04-30T06:44:09.535ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
113072025-04-30T06:44:09.535ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
113082025-04-30T06:44:09.535ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
113092025-04-30T06:44:09.536ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
113102025-04-30T06:44:09.538ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
113112025-04-30T06:44:09.538ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
113122025-04-30T06:44:09.538ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
113132025-04-30T06:44:09.608ZWARNtest_unauthorized (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
113142025-04-30T06:44:09.609ZWARNtest_unauthorized (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
113152025-04-30T06:44:09.609ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 991.159441ms iteration = 39 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
113162025-04-30T06:44:09.609ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 40 reason = Signaled
113172025-04-30T06:44:09.609ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
113182025-04-30T06:44:09.609ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
113192025-04-30T06:44:09.609ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
113202025-04-30T06:44:09.609ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
113212025-04-30T06:44:09.611ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
113222025-04-30T06:44:09.618ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
113232025-04-30T06:44:09.619ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113242025-04-30T06:44:09.626ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
113252025-04-30T06:44:09.626ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
113262025-04-30T06:44:09.626ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
113272025-04-30T06:44:09.627ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
113282025-04-30T06:44:09.631ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1a8b2318-abf8-406d-a389-112ab91a0de8, 0)]
113292025-04-30T06:44:09.631ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
113302025-04-30T06:44:09.634ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 99 8e 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:58985 req_id = 6163c25a-6746-44e0-92c1-3240f3b82a23
113312025-04-30T06:44:09.635ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 39310,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:58985 req_id = 6163c25a-6746-44e0-92c1-3240f3b82a23
113322025-04-30T06:44:09.635ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
113332025-04-30T06:44:09.635ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
113342025-04-30T06:44:09.646ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:58985 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 6163c25a-6746-44e0-92c1-3240f3b82a23
113352025-04-30T06:44:09.652ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4314, tv_nsec: 450830239 } })
113362025-04-30T06:44:09.652ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
113372025-04-30T06:44:09.652ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 21 65 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e !e..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:63357 req_id = 2154b2ae-4f65-4dc4-9c4e-52ec8abb1d57
113382025-04-30T06:44:09.652ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 8549,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63357 req_id = 2154b2ae-4f65-4dc4-9c4e-52ec8abb1d57
113392025-04-30T06:44:09.652ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
113402025-04-30T06:44:09.652ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:63357 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 2154b2ae-4f65-4dc4-9c4e-52ec8abb1d57
113412025-04-30T06:44:09.710ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
113422025-04-30T06:44:09.711ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
113432025-04-30T06:44:09.711ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
113442025-04-30T06:44:09.711ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
113452025-04-30T06:44:09.711ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
113462025-04-30T06:44:09.713ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
113472025-04-30T06:44:09.714ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 06 b2 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:52042 req_id = ec66034e-527c-43d1-bbbc-08e24ca7db8a
113482025-04-30T06:44:09.714ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 1714,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52042 req_id = ec66034e-527c-43d1-bbbc-08e24ca7db8a
113492025-04-30T06:44:09.714ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
113502025-04-30T06:44:09.714ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 1714, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:52042 req_id = ec66034e-527c-43d1-bbbc-08e24ca7db8a
113512025-04-30T06:44:09.714ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
113522025-04-30T06:44:09.714ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
113532025-04-30T06:44:09.714ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
113542025-04-30T06:44:09.716ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113552025-04-30T06:44:09.721ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
113562025-04-30T06:44:09.721ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
113572025-04-30T06:44:09.721ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
113582025-04-30T06:44:09.722ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
113592025-04-30T06:44:09.723ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "7bf6e75c-0a05-455d-8237-0538a8259dff", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
113602025-04-30T06:44:09.723ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
113612025-04-30T06:44:09.725ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
113622025-04-30T06:44:09.725ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
113632025-04-30T06:44:09.727ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
113642025-04-30T06:44:09.727ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6ff043d9-3b81-48a1-a987-cb869983a4a8", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
113652025-04-30T06:44:09.727ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
113662025-04-30T06:44:09.730ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "88c50a0d-b00d-49bc-881c-67d072d2a21b", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
113672025-04-30T06:44:09.730ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
113682025-04-30T06:44:09.733ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "77c4aea6-e828-404f-a515-eb35c553810d", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
113692025-04-30T06:44:09.733ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
113702025-04-30T06:44:09.735ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "35625f7a-f35e-46f3-9820-137ccfe6ebca", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
113712025-04-30T06:44:09.735ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
113722025-04-30T06:44:09.746ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b1bd3b52-b92d-4d6c-b4d3-cbe51f0860b3", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
113732025-04-30T06:44:09.746ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
113742025-04-30T06:44:09.750ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "156f66a4-1083-487d-9e72-26485dc098de", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
113752025-04-30T06:44:09.750ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
113762025-04-30T06:44:09.753ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "20c51456-3b8c-471b-afe0-451a1b834ecc", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
113772025-04-30T06:44:09.753ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
113782025-04-30T06:44:09.755ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b0ca7bc8-9e84-4de5-9ea3-0ce6ec34d055", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
113792025-04-30T06:44:09.755ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
113802025-04-30T06:44:09.760ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "04ca646d-ee4b-4711-8da9-c376cdc3b52a", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
113812025-04-30T06:44:09.760ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
113822025-04-30T06:44:09.761ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
113832025-04-30T06:44:09.764ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e625b6e2-8345-40fc-aacd-93ea2dea928b", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
113842025-04-30T06:44:09.764ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
113852025-04-30T06:44:09.765ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5ecf1eda-86db-4a70-91cc-ac8d1a135877", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
113862025-04-30T06:44:09.765ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
113872025-04-30T06:44:09.767ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "48e72834-bb33-4197-9fa7-fcc43cce7931", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
113882025-04-30T06:44:09.767ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
113892025-04-30T06:44:09.770ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a361582c-6052-4775-8390-edd2636a0c5f", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
113902025-04-30T06:44:09.770ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
113912025-04-30T06:44:09.772ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e41e860f-2ed1-4ba1-ad94-76976f8c3dc1", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
113922025-04-30T06:44:09.772ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
113932025-04-30T06:44:09.775ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e048ddae-4c17-4eb3-9ade-60d88b3d5194", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
113942025-04-30T06:44:09.775ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
113952025-04-30T06:44:09.777ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "39640967-aeb3-4a4f-88e4-df617f08f683", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
113962025-04-30T06:44:09.778ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
113972025-04-30T06:44:09.781ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7772927e-2397-45ec-bc2e-80d0fa781878", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
113982025-04-30T06:44:09.782ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
113992025-04-30T06:44:09.784ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8e161293-8485-4677-9b11-a241bd6a4e38", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
114002025-04-30T06:44:09.784ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
114012025-04-30T06:44:09.785ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "949e9e0d-8260-499c-9bf5-6a351c0703b4", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
114022025-04-30T06:44:09.785ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
114032025-04-30T06:44:09.787ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6f6f3152-375c-406e-8f77-a3bb7131312c", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
114042025-04-30T06:44:09.788ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
114052025-04-30T06:44:09.789ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b0258328-4072-4198-b8b0-8b430412a12b", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
114062025-04-30T06:44:09.789ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
114072025-04-30T06:44:09.790ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "60448eef-69d9-4680-b3b8-2e513891ed70", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
114082025-04-30T06:44:09.790ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
114092025-04-30T06:44:09.792ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "78e5985f-c452-4577-9a55-09fbf5c44bd7", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
114102025-04-30T06:44:09.792ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
114112025-04-30T06:44:09.794ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc531fdf-216f-47bd-824d-b7125bbd966b", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
114122025-04-30T06:44:09.794ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
114132025-04-30T06:44:09.797ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "45104754-b7b5-49f3-ac49-75da2c44a80a", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
114142025-04-30T06:44:09.798ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
114152025-04-30T06:44:09.800ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b64a260d-10ec-4857-be77-f69312584df4", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
114162025-04-30T06:44:09.800ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
114172025-04-30T06:44:09.802ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "64a5ea27-56ff-49e5-ad97-94d9eb42e7a8", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
114182025-04-30T06:44:09.802ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
114192025-04-30T06:44:09.804ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "09202af1-17a5-4e34-bfa1-5df69f552616", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
114202025-04-30T06:44:09.804ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
114212025-04-30T06:44:09.807ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "38042546-8ebb-43a0-9e34-bfbef2f5ee4f", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
114222025-04-30T06:44:09.807ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
114232025-04-30T06:44:09.808ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3e93e3ce-6716-4b93-841a-ebc22409f52a", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
114242025-04-30T06:44:09.808ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
114252025-04-30T06:44:09.809ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6e72463e-5b65-48b8-abee-074c7ed73031", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
114262025-04-30T06:44:09.809ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
114272025-04-30T06:44:09.810ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "77e52489-14c6-49db-9742-fd6462cc520a", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
114282025-04-30T06:44:09.810ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
114292025-04-30T06:44:09.813ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "31055a30-cbe5-4902-9e65-30523dc0e6ad", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
114302025-04-30T06:44:09.813ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
114312025-04-30T06:44:09.813ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
114322025-04-30T06:44:09.813ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
114332025-04-30T06:44:09.813ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
114342025-04-30T06:44:09.813ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
114352025-04-30T06:44:09.814ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
114362025-04-30T06:44:09.817ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ce52611c-7495-4856-9191-303f19b644a7", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
114372025-04-30T06:44:09.817ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
114382025-04-30T06:44:09.818ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114392025-04-30T06:44:09.827ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
114402025-04-30T06:44:09.827ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
114412025-04-30T06:44:09.827ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
114422025-04-30T06:44:09.828ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
114432025-04-30T06:44:09.831ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
114442025-04-30T06:44:09.831ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
114452025-04-30T06:44:09.832ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
114462025-04-30T06:44:09.832ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "360a26e3-49ce-4b19-9ad6-e3cfe9f65f63", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
114472025-04-30T06:44:09.832ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
114482025-04-30T06:44:09.843ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "92a1208b-5da7-4937-a8c4-702cfef4628e", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
114492025-04-30T06:44:09.844ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
114502025-04-30T06:44:09.849ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "706f795b-e8ef-46b8-90a2-427b6927c8be", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
114512025-04-30T06:44:09.849ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
114522025-04-30T06:44:09.851ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4da23b6a-28f3-4b7c-bac8-978e83d5c225", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
114532025-04-30T06:44:09.851ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
114542025-04-30T06:44:09.863ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0c570c1-4cb5-4c0b-8d37-3911070a6de1", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
114552025-04-30T06:44:09.863ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
114562025-04-30T06:44:09.871ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
114572025-04-30T06:44:09.874ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "aed3f7ee-9dd5-4d85-a267-abd8359ab2aa", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
114582025-04-30T06:44:09.875ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
114592025-04-30T06:44:09.880ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "255c0007-2218-4083-b191-c18e86150c39", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
114602025-04-30T06:44:09.881ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
114612025-04-30T06:44:09.884ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e7c9d8b2-aa57-4b07-96f8-fb7469b3f7a6", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
114622025-04-30T06:44:09.887ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
114632025-04-30T06:44:09.889ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "aa6d1abc-577e-443e-a7bf-c27b8ece8ae1", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
114642025-04-30T06:44:09.889ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
114652025-04-30T06:44:09.891ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3345ae0d-bd51-4d7c-8b21-53048e640bee", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
114662025-04-30T06:44:09.892ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
114672025-04-30T06:44:09.892ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
114682025-04-30T06:44:09.892ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
114692025-04-30T06:44:09.892ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
114702025-04-30T06:44:09.892ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
114712025-04-30T06:44:09.896ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114722025-04-30T06:44:09.903ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
114732025-04-30T06:44:09.903ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
114742025-04-30T06:44:09.903ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
114752025-04-30T06:44:09.903ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
114762025-04-30T06:44:09.903ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
114772025-04-30T06:44:09.903ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
114782025-04-30T06:44:09.914ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114792025-04-30T06:44:09.920ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
114802025-04-30T06:44:09.920ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
114812025-04-30T06:44:09.920ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
114822025-04-30T06:44:09.920ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
114832025-04-30T06:44:09.922ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
114842025-04-30T06:44:09.922ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
114852025-04-30T06:44:09.922ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
114862025-04-30T06:44:09.923ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
114872025-04-30T06:44:09.925ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
114882025-04-30T06:44:09.977ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
114892025-04-30T06:44:09.977ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory
114902025-04-30T06:44:09.979ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:34146 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
114912025-04-30T06:44:09.979ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114922025-04-30T06:44:09.988ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
114932025-04-30T06:44:09.988ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
114942025-04-30T06:44:09.989ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
114952025-04-30T06:44:09.990ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
114962025-04-30T06:44:09.990ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:34146 req_id = a03cebf2-73c5-4177-8637-ede6e2c662f1 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
114972025-04-30T06:44:09.992ZINFOtest_unauthorized (dropshot): request completed latency_us = 2009 local_addr = [::1]:57786 method = GET remote_addr = [::1]:34146 req_id = a03cebf2-73c5-4177-8637-ede6e2c662f1 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
114982025-04-30T06:44:09.994ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = http://[::1]:57786 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:57786/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "a03cebf2-73c5-4177-8637-ede6e2c662f1", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
114992025-04-30T06:44:09.998ZDEBGtest_unauthorized (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
115002025-04-30T06:44:09.998ZDEBGtest_unauthorized (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
115012025-04-30T06:44:09.998ZDEBGtest_unauthorized (BackgroundTasks): finished collection background_task = inventory_collection
115022025-04-30T06:44:10.001ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
115032025-04-30T06:44:10.001ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
115042025-04-30T06:44:10.002ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
115052025-04-30T06:44:10.002ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
115062025-04-30T06:44:10.004ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
115072025-04-30T06:44:10.007ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
115082025-04-30T06:44:10.007ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
115092025-04-30T06:44:10.008ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
115102025-04-30T06:44:10.008ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115112025-04-30T06:44:10.014ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
115122025-04-30T06:44:10.015ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
115132025-04-30T06:44:10.015ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
115142025-04-30T06:44:10.015ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
115152025-04-30T06:44:10.062ZWARNtest_unauthorized (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
115162025-04-30T06:44:10.062ZWARNtest_unauthorized (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
115172025-04-30T06:44:10.062ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 453.218007ms iteration = 40 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
115182025-04-30T06:44:10.064ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 41 reason = Signaled
115192025-04-30T06:44:10.064ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
115202025-04-30T06:44:10.064ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
115212025-04-30T06:44:10.064ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
115222025-04-30T06:44:10.064ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
115232025-04-30T06:44:10.071ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
115242025-04-30T06:44:10.072ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
115252025-04-30T06:44:10.093ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115262025-04-30T06:44:10.124ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
115272025-04-30T06:44:10.126ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
115282025-04-30T06:44:10.126ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
115292025-04-30T06:44:10.127ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
115302025-04-30T06:44:10.134ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1a8b2318-abf8-406d-a389-112ab91a0de8, 0)]
115312025-04-30T06:44:10.134ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
115322025-04-30T06:44:10.136ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 4a ce 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 J............_mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:65071 req_id = 67839b64-cebb-4adb-91fa-ba71faee6db9
115332025-04-30T06:44:10.136ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 19150,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:65071 req_id = 67839b64-cebb-4adb-91fa-ba71faee6db9
115342025-04-30T06:44:10.137ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
115352025-04-30T06:44:10.137ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
115362025-04-30T06:44:10.137ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:65071 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 67839b64-cebb-4adb-91fa-ba71faee6db9
115372025-04-30T06:44:10.138ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4314, tv_nsec: 937046042 } })
115382025-04-30T06:44:10.138ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
115392025-04-30T06:44:10.138ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 44 3e 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e D>..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:62137 req_id = ef2155cd-aede-4cb5-8ed7-1a422b44f8df
115402025-04-30T06:44:10.138ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 17470,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62137 req_id = ef2155cd-aede-4cb5-8ed7-1a422b44f8df
115412025-04-30T06:44:10.139ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
115422025-04-30T06:44:10.139ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:62137 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = ef2155cd-aede-4cb5-8ed7-1a422b44f8df
115432025-04-30T06:44:10.139ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
115442025-04-30T06:44:10.139ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
115452025-04-30T06:44:10.140ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
115462025-04-30T06:44:10.140ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
115472025-04-30T06:44:10.141ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
115482025-04-30T06:44:10.211ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
115492025-04-30T06:44:10.213ZDEBGtest_unauthorized (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
115502025-04-30T06:44:10.213ZDEBGtest_unauthorized (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
115512025-04-30T06:44:10.214ZDEBGtest_unauthorized (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
115522025-04-30T06:44:10.214ZDEBGtest_unauthorized (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
115532025-04-30T06:44:10.215ZDEBGtest_unauthorized (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a
115542025-04-30T06:44:10.216ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
115552025-04-30T06:44:10.218ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: e4 2d 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c .-..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:58538 req_id = f29ab159-473d-406b-bef9-28ea8f76598d
115562025-04-30T06:44:10.223ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 58413,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:58538 req_id = f29ab159-473d-406b-bef9-28ea8f76598d
115572025-04-30T06:44:10.223ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
115582025-04-30T06:44:10.223ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 58413, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:58538 req_id = f29ab159-473d-406b-bef9-28ea8f76598d
115592025-04-30T06:44:10.225ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115602025-04-30T06:44:10.236ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
115612025-04-30T06:44:10.236ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
115622025-04-30T06:44:10.236ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
115632025-04-30T06:44:10.237ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
115642025-04-30T06:44:10.238ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
115652025-04-30T06:44:10.238ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
115662025-04-30T06:44:10.238ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
115672025-04-30T06:44:10.238ZTRACtest_unauthorized (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
115682025-04-30T06:44:10.239ZDEBGtest_unauthorized (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
115692025-04-30T06:44:10.239ZTRACtest_unauthorized (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
115702025-04-30T06:44:10.239ZTRACtest_unauthorized (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
115712025-04-30T06:44:10.241ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 2 table_name = fields_string
115722025-04-30T06:44:10.241ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
115732025-04-30T06:44:10.251ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
115742025-04-30T06:44:10.251ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
115752025-04-30T06:44:10.252ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
115762025-04-30T06:44:10.260ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 1 table_name = measurements_i64
115772025-04-30T06:44:10.260ZTRACtest_unauthorized (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
115782025-04-30T06:44:10.260ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "0435de61-2b6b-41e6-a667-08cc62a839c7", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
115792025-04-30T06:44:10.260ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
115802025-04-30T06:44:10.266ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bd72b2a9-a961-49c6-83cc-3be5f9dd4392", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
115812025-04-30T06:44:10.266ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
115822025-04-30T06:44:10.268ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "84d1ff6c-9e16-4a68-aab9-17099d98a099", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
115832025-04-30T06:44:10.268ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
115842025-04-30T06:44:10.270ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fe6deaf4-ad5b-4d0d-87f2-5722e838ac46", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
115852025-04-30T06:44:10.270ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
115862025-04-30T06:44:10.275ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1e3f6234-474a-471d-a94f-678a9667a359", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
115872025-04-30T06:44:10.276ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
115882025-04-30T06:44:10.280ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cbc7e135-e979-428c-92aa-56f1623717e4", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
115892025-04-30T06:44:10.282ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
115902025-04-30T06:44:10.284ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b5c741d7-ce7f-4930-94fd-9bd9a39405fa", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
115912025-04-30T06:44:10.284ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
115922025-04-30T06:44:10.287ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ad2bc8a9-b4d7-4093-a8f0-cd8e0902ef01", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
115932025-04-30T06:44:10.287ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
115942025-04-30T06:44:10.288ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cfa20a1a-c6cd-4145-b322-9c06aa20c241", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
115952025-04-30T06:44:10.288ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
115962025-04-30T06:44:10.289ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "889a26b3-ad7a-4b9b-9efd-4eb098fb2255", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
115972025-04-30T06:44:10.289ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
115982025-04-30T06:44:10.291ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c53a6908-54ae-40f1-90a6-c39784a56e93", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
115992025-04-30T06:44:10.291ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
116002025-04-30T06:44:10.292ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "25e21947-f68c-43c3-b9b5-0a161674808d", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
116012025-04-30T06:44:10.292ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
116022025-04-30T06:44:10.294ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4eb394c4-1a70-452a-98e6-0ceec69c6a24", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
116032025-04-30T06:44:10.294ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
116042025-04-30T06:44:10.296ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e50907dd-a7a7-4625-8e40-5b060e76ee33", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
116052025-04-30T06:44:10.296ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
116062025-04-30T06:44:10.303ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3330fd78-b6ed-4811-8751-5d3a66e96849", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
116072025-04-30T06:44:10.303ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
116082025-04-30T06:44:10.307ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1463c9e3-c5cb-424f-b38f-de9bb645f9ef", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
116092025-04-30T06:44:10.307ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
116102025-04-30T06:44:10.308ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
116112025-04-30T06:44:10.311ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5de741ce-1223-49da-95c0-9e2e593afb7f", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
116122025-04-30T06:44:10.311ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
116132025-04-30T06:44:10.314ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bc0f0e3b-563d-4be5-aade-28b9296185fe", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
116142025-04-30T06:44:10.314ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
116152025-04-30T06:44:10.315ZWARNtest_unauthorized (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
116162025-04-30T06:44:10.316ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6b2afe01-1dba-4fcb-89c8-c9035a0d84b2", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
116172025-04-30T06:44:10.317ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
116182025-04-30T06:44:10.319ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "46a1e353-f8d2-4534-b2a7-539931ad570e", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
116192025-04-30T06:44:10.319ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
116202025-04-30T06:44:10.321ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "775ab791-f398-4b6b-a45c-a3bd08958aa6", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
116212025-04-30T06:44:10.321ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
116222025-04-30T06:44:10.322ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "02e082b6-01e0-4abd-ab31-37847f23a010", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
116232025-04-30T06:44:10.323ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
116242025-04-30T06:44:10.324ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4a9275f6-59be-4df4-a1cc-87f3fae0ba02", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
116252025-04-30T06:44:10.324ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
116262025-04-30T06:44:10.326ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0a841b17-1728-4c13-ae06-442f1556fbe6", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
116272025-04-30T06:44:10.327ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
116282025-04-30T06:44:10.330ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "15e80a89-0acd-4f41-aa84-9041b5002b1d", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
116292025-04-30T06:44:10.330ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
116302025-04-30T06:44:10.352ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "86eb6e91-031e-4f1c-82b3-63ca1985a610", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
116312025-04-30T06:44:10.353ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
116322025-04-30T06:44:10.359ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
116332025-04-30T06:44:10.360ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
116342025-04-30T06:44:10.360ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
116352025-04-30T06:44:10.360ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
116362025-04-30T06:44:10.362ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
116372025-04-30T06:44:10.366ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ad6fccbb-b715-4269-a6e0-588ba8fc6ce3", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
116382025-04-30T06:44:10.367ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
116392025-04-30T06:44:10.367ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116402025-04-30T06:44:10.379ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
116412025-04-30T06:44:10.379ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
116422025-04-30T06:44:10.379ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
116432025-04-30T06:44:10.381ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
116442025-04-30T06:44:10.384ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "34e9d80d-593a-4494-8900-3f13e39c5d74", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
116452025-04-30T06:44:10.384ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
116462025-04-30T06:44:10.391ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "486643aa-e6c6-4e4f-bf52-4fa2c5c01893", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
116472025-04-30T06:44:10.392ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
116482025-04-30T06:44:10.398ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
116492025-04-30T06:44:10.398ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
116502025-04-30T06:44:10.399ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
116512025-04-30T06:44:10.410ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "803dae85-4488-4e46-b600-83fb24bd0474", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
116522025-04-30T06:44:10.411ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
116532025-04-30T06:44:10.412ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e20e61e5-e121-4253-93ec-e4bd088e4e2b", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
116542025-04-30T06:44:10.412ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
116552025-04-30T06:44:10.414ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a43e4a57-7ba8-4a59-aa3e-3d5011226b27", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
116562025-04-30T06:44:10.415ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
116572025-04-30T06:44:10.416ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4f57735e-e9ae-42ae-a264-f439fb27a7a1", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
116582025-04-30T06:44:10.416ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
116592025-04-30T06:44:10.417ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "751bcb60-ca58-444b-9918-20cad88a000d", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
116602025-04-30T06:44:10.417ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
116612025-04-30T06:44:10.419ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9bc72d20-93dd-4125-a149-8e5a34009d02", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
116622025-04-30T06:44:10.419ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
116632025-04-30T06:44:10.423ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0c2803af-72ab-4317-ac20-b9e96380bd88", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
116642025-04-30T06:44:10.423ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
116652025-04-30T06:44:10.431ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "15861fae-f804-46c8-a818-f439c047d1bf", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
116662025-04-30T06:44:10.431ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
116672025-04-30T06:44:10.434ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "70f50eb3-16e2-4e4c-99fc-464c646d7ad1", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
116682025-04-30T06:44:10.434ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
116692025-04-30T06:44:10.437ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f3cd6446-5212-4f2f-bb2d-1db35eb84887", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
116702025-04-30T06:44:10.437ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
116712025-04-30T06:44:10.439ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0da219a9-f054-4e56-8db4-3ab7ad0940cd", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
116722025-04-30T06:44:10.439ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
116732025-04-30T06:44:10.443ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6137cb98-2088-4030-a898-8a0c5c9120f8", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
116742025-04-30T06:44:10.443ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
116752025-04-30T06:44:10.444ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "09d89f47-bb29-4c04-815c-36a9edd3b086", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
116762025-04-30T06:44:10.444ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
116772025-04-30T06:44:10.451ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb579786-cbeb-4410-b1af-0d6a2bac5f66", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
116782025-04-30T06:44:10.452ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
116792025-04-30T06:44:10.453ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0d89ffb7-5b7b-43a3-b5e3-8fc8820296c6", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
116802025-04-30T06:44:10.453ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
116812025-04-30T06:44:10.453ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f75c2471-ce8b-4d6a-80f2-1d9c12092a50", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
116822025-04-30T06:44:10.453ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
116832025-04-30T06:44:10.454ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
116842025-04-30T06:44:10.454ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
116852025-04-30T06:44:10.454ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
116862025-04-30T06:44:10.454ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
116872025-04-30T06:44:10.467ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116882025-04-30T06:44:10.474ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
116892025-04-30T06:44:10.474ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
116902025-04-30T06:44:10.474ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
116912025-04-30T06:44:10.474ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
116922025-04-30T06:44:10.474ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
116932025-04-30T06:44:10.474ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
116942025-04-30T06:44:10.477ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116952025-04-30T06:44:10.493ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
116962025-04-30T06:44:10.499ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
116972025-04-30T06:44:10.499ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
116982025-04-30T06:44:10.500ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
116992025-04-30T06:44:10.522ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
117002025-04-30T06:44:10.576ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
117012025-04-30T06:44:10.576ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
117022025-04-30T06:44:10.576ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
117032025-04-30T06:44:10.576ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
117042025-04-30T06:44:10.577ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
117052025-04-30T06:44:10.584ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117062025-04-30T06:44:10.590ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
117072025-04-30T06:44:10.590ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
117082025-04-30T06:44:10.590ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
117092025-04-30T06:44:10.592ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
117102025-04-30T06:44:10.594ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
117112025-04-30T06:44:10.594ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
117122025-04-30T06:44:10.599ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
117132025-04-30T06:44:10.700ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
117142025-04-30T06:44:10.700ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory
117152025-04-30T06:44:10.701ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:55196 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
117162025-04-30T06:44:10.702ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:55196 req_id = 27a9a3d5-175e-4b7e-af19-c30e165580a9 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
117172025-04-30T06:44:10.709ZINFOtest_unauthorized (dropshot): request completed latency_us = 7243 local_addr = [::1]:57786 method = GET remote_addr = [::1]:55196 req_id = 27a9a3d5-175e-4b7e-af19-c30e165580a9 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
117182025-04-30T06:44:10.710ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = http://[::1]:57786 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:57786/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "27a9a3d5-175e-4b7e-af19-c30e165580a9", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
117192025-04-30T06:44:10.715ZDEBGtest_unauthorized (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
117202025-04-30T06:44:10.715ZDEBGtest_unauthorized (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
117212025-04-30T06:44:10.715ZDEBGtest_unauthorized (BackgroundTasks): finished collection background_task = inventory_collection
117222025-04-30T06:44:10.717ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
117232025-04-30T06:44:10.717ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
117242025-04-30T06:44:10.717ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
117252025-04-30T06:44:10.718ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
117262025-04-30T06:44:10.718ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
117272025-04-30T06:44:10.723ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117282025-04-30T06:44:10.731ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
117292025-04-30T06:44:10.732ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
117302025-04-30T06:44:10.732ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
117312025-04-30T06:44:10.735ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
117322025-04-30T06:44:10.769ZTRACtest_unauthorized (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
117332025-04-30T06:44:10.777ZTRACtest_unauthorized (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 42 e0 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 B............_de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:54073 req_id = 6bf45ab1-03bc-46ca-b96c-5c881018dda1
117342025-04-30T06:44:10.777ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 17120,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54073 req_id = 6bf45ab1-03bc-46ca-b96c-5c881018dda1
117352025-04-30T06:44:10.777ZDEBGtest_unauthorized (store): query key key = _dendrite._tcp
117362025-04-30T06:44:10.778ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
117372025-04-30T06:44:10.778ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:54073 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 44264, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 6bf45ab1-03bc-46ca-b96c-5c881018dda1
117382025-04-30T06:44:10.779ZDEBGtest_unauthorized (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 44264, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4315, tv_nsec: 578250651 } })
117392025-04-30T06:44:10.779ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
117402025-04-30T06:44:10.780ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: e7 0a 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:63393 req_id = 69682bb2-fff5-4aa0-bade-896cfd4b33c8
117412025-04-30T06:44:10.781ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 59146,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63393 req_id = 69682bb2-fff5-4aa0-bade-896cfd4b33c8
117422025-04-30T06:44:10.781ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
117432025-04-30T06:44:10.781ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:63393 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 69682bb2-fff5-4aa0-bade-896cfd4b33c8
117442025-04-30T06:44:10.782ZINFOtest_unauthorized (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector
117452025-04-30T06:44:10.892ZINFOtest_unauthorized (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector zone_address = ::1
117462025-04-30T06:44:10.892ZDEBGtest_unauthorized (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[::1]:12225/local/switch-id
117472025-04-30T06:44:10.893ZDEBGtest_unauthorized (MgsClient): client response background_task = nat_v4_garbage_collector result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
117482025-04-30T06:44:10.893ZWARNtest_unauthorized (BackgroundTasks): failed to identify switch slot for dendrite background_task = nat_v4_garbage_collector reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
117492025-04-30T06:44:10.894ZWARNtest_unauthorized (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = nat_v4_garbage_collector
117502025-04-30T06:44:10.894ZTRACtest_unauthorized (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
117512025-04-30T06:44:10.895ZTRACtest_unauthorized (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 5c 81 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 \\............_de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:61587 req_id = 623d2690-6c61-4266-b851-b9d2dd4c6257
117522025-04-30T06:44:10.895ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 23681,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:61587 req_id = 623d2690-6c61-4266-b851-b9d2dd4c6257
117532025-04-30T06:44:10.895ZDEBGtest_unauthorized (store): query key key = _dendrite._tcp
117542025-04-30T06:44:10.896ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
117552025-04-30T06:44:10.896ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:61587 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 44264, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 623d2690-6c61-4266-b851-b9d2dd4c6257
117562025-04-30T06:44:10.896ZDEBGtest_unauthorized (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 44264, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4315, tv_nsec: 695817424 } })
117572025-04-30T06:44:10.896ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
117582025-04-30T06:44:10.897ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: ab f9 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:63717 req_id = 6f767c9c-c001-4873-8f65-3be3ff8e04ec
117592025-04-30T06:44:10.898ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 44025,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63717 req_id = 6f767c9c-c001-4873-8f65-3be3ff8e04ec
117602025-04-30T06:44:10.898ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
117612025-04-30T06:44:10.898ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:63717 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 6f767c9c-c001-4873-8f65-3be3ff8e04ec
117622025-04-30T06:44:10.898ZINFOtest_unauthorized (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager
117632025-04-30T06:44:10.959ZINFOtest_unauthorized (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager zone_address = ::1
117642025-04-30T06:44:10.960ZDEBGtest_unauthorized (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[::1]:12225/local/switch-id
117652025-04-30T06:44:10.961ZDEBGtest_unauthorized (MgsClient): client response background_task = bfd_manager result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
117662025-04-30T06:44:10.961ZWARNtest_unauthorized (BackgroundTasks): failed to identify switch slot for dendrite background_task = bfd_manager reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
117672025-04-30T06:44:10.964ZWARNtest_unauthorized (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = bfd_manager
117682025-04-30T06:44:10.969ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
117692025-04-30T06:44:10.974ZWARNtest_unauthorized (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
117702025-04-30T06:44:10.974ZWARNtest_unauthorized (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
117712025-04-30T06:44:10.974ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 910.261336ms iteration = 41 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
117722025-04-30T06:44:10.974ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 42 reason = Signaled
117732025-04-30T06:44:10.974ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
117742025-04-30T06:44:10.974ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
117752025-04-30T06:44:10.974ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
117762025-04-30T06:44:10.975ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
117772025-04-30T06:44:10.975ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
117782025-04-30T06:44:10.979ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117792025-04-30T06:44:10.985ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
117802025-04-30T06:44:10.985ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
117812025-04-30T06:44:10.985ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
117822025-04-30T06:44:10.986ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
117832025-04-30T06:44:10.989ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1a8b2318-abf8-406d-a389-112ab91a0de8, 0)]
117842025-04-30T06:44:10.990ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
117852025-04-30T06:44:10.990ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 54 1e 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 T............_mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:64000 req_id = 92d23849-1d74-448f-837e-f6a975bfff45
117862025-04-30T06:44:10.992ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 21534,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64000 req_id = 92d23849-1d74-448f-837e-f6a975bfff45
117872025-04-30T06:44:10.992ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
117882025-04-30T06:44:10.992ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
117892025-04-30T06:44:10.992ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:64000 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 92d23849-1d74-448f-837e-f6a975bfff45
117902025-04-30T06:44:10.993ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4315, tv_nsec: 792377805 } })
117912025-04-30T06:44:10.993ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
117922025-04-30T06:44:10.993ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 68 ed 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e h...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:52658 req_id = 1b124284-18f0-47b8-ac63-bc507d0d2ecd
117932025-04-30T06:44:10.994ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 26861,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52658 req_id = 1b124284-18f0-47b8-ac63-bc507d0d2ecd
117942025-04-30T06:44:10.994ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
117952025-04-30T06:44:10.994ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:52658 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 1b124284-18f0-47b8-ac63-bc507d0d2ecd
117962025-04-30T06:44:11.084ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
117972025-04-30T06:44:11.085ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
117982025-04-30T06:44:11.085ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
117992025-04-30T06:44:11.085ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
118002025-04-30T06:44:11.085ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
118012025-04-30T06:44:11.088ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
118022025-04-30T06:44:11.089ZTRACtest_unauthorized (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
118032025-04-30T06:44:11.091ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 5e ca 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ^............_cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:63729 req_id = 75f35a44-0812-4d34-8bba-638d34ca8a21
118042025-04-30T06:44:11.092ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 24266,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63729 req_id = 75f35a44-0812-4d34-8bba-638d34ca8a21
118052025-04-30T06:44:11.093ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
118062025-04-30T06:44:11.093ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 24266, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:63729 req_id = 75f35a44-0812-4d34-8bba-638d34ca8a21
118072025-04-30T06:44:11.093ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
118082025-04-30T06:44:11.093ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
118092025-04-30T06:44:11.093ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
118102025-04-30T06:44:11.093ZTRACtest_unauthorized (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 5b 3f 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 [?..........._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:53814 req_id = 636fdc1e-00c5-420b-bd3a-9183e1a0e199
118112025-04-30T06:44:11.093ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 23359,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:53814 req_id = 636fdc1e-00c5-420b-bd3a-9183e1a0e199
118122025-04-30T06:44:11.094ZDEBGtest_unauthorized (store): query key key = _dendrite._tcp
118132025-04-30T06:44:11.094ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
118142025-04-30T06:44:11.094ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:53814 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 44264, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 636fdc1e-00c5-420b-bd3a-9183e1a0e199
118152025-04-30T06:44:11.094ZDEBGtest_unauthorized (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 44264, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4315, tv_nsec: 893723820 } })
118162025-04-30T06:44:11.094ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
118172025-04-30T06:44:11.096ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: af 58 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .X..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:57521 req_id = de097ee1-9305-4fc0-9897-38d49bb3df4f
118182025-04-30T06:44:11.096ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 44888,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:57521 req_id = de097ee1-9305-4fc0-9897-38d49bb3df4f
118192025-04-30T06:44:11.096ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
118202025-04-30T06:44:11.096ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:57521 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = de097ee1-9305-4fc0-9897-38d49bb3df4f
118212025-04-30T06:44:11.096ZINFOtest_unauthorized (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc
118222025-04-30T06:44:11.152ZINFOtest_unauthorized (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc zone_address = ::1
118232025-04-30T06:44:11.152ZDEBGtest_unauthorized (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc uri = http://[::1]:12225/local/switch-id
118242025-04-30T06:44:11.152ZDEBGtest_unauthorized (MgsClient): client response background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
118252025-04-30T06:44:11.152ZWARNtest_unauthorized (BackgroundTasks): failed to identify switch slot for dendrite background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
118262025-04-30T06:44:11.154ZWARNtest_unauthorized (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc
118272025-04-30T06:44:11.159ZINFOtest_unauthorized (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
118282025-04-30T06:44:11.160ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa25fd39-ca8b-4c3e-b7cd-33959e74963e", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
118292025-04-30T06:44:11.160ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
118302025-04-30T06:44:11.161ZDEBGtest_unauthorized (oximeter): client request body = None method = GET uri = http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
118312025-04-30T06:44:11.162ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118322025-04-30T06:44:11.178ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
118332025-04-30T06:44:11.178ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
118342025-04-30T06:44:11.179ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
118352025-04-30T06:44:11.179ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
118362025-04-30T06:44:11.179ZTRACtest_unauthorized (dropshot_internal): incoming request local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 7075ee4e-e232-42d4-9f77-4d37c44950f3 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
118372025-04-30T06:44:11.181ZTRACtest_unauthorized (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 7075ee4e-e232-42d4-9f77-4d37c44950f3 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
118382025-04-30T06:44:11.181ZDEBGtest_unauthorized (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 7075ee4e-e232-42d4-9f77-4d37c44950f3 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
118392025-04-30T06:44:11.184ZDEBGtest_unauthorized (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 7075ee4e-e232-42d4-9f77-4d37c44950f3 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
118402025-04-30T06:44:11.185ZDEBGtest_unauthorized (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
118412025-04-30T06:44:11.185ZDEBGtest_unauthorized (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
118422025-04-30T06:44:11.185ZDEBGtest_unauthorized (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a
118432025-04-30T06:44:11.187ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
118442025-04-30T06:44:11.194ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
118452025-04-30T06:44:11.194ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
118462025-04-30T06:44:11.195ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
118472025-04-30T06:44:11.195ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 2 table_name = fields_string
118482025-04-30T06:44:11.195ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
118492025-04-30T06:44:11.199ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 1 table_name = measurements_i64
118502025-04-30T06:44:11.199ZTRACtest_unauthorized (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
118512025-04-30T06:44:11.199ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c4d9f5db-075b-4c8a-8a94-950b622ab3f6", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
118522025-04-30T06:44:11.200ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
118532025-04-30T06:44:11.200ZINFOtest_unauthorized (dropshot_internal): request completed latency_us = 20927 local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 7075ee4e-e232-42d4-9f77-4d37c44950f3 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
118542025-04-30T06:44:11.202ZDEBGtest_unauthorized (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
118552025-04-30T06:44:11.202ZDEBGtest_unauthorized (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "7075ee4e-e232-42d4-9f77-4d37c44950f3", "content-length": "405", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
118562025-04-30T06:44:11.202ZDEBGtest_unauthorized (oximeter): client request body = None method = GET uri = http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
118572025-04-30T06:44:11.203ZDEBGtest_unauthorized (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
118582025-04-30T06:44:11.205ZTRACtest_unauthorized (dropshot_internal): incoming request local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = da992886-424e-41fe-ae8d-cefa7719ea34 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
118592025-04-30T06:44:11.207ZTRACtest_unauthorized (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = da992886-424e-41fe-ae8d-cefa7719ea34 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
118602025-04-30T06:44:11.207ZDEBGtest_unauthorized (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = da992886-424e-41fe-ae8d-cefa7719ea34 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
118612025-04-30T06:44:11.210ZDEBGtest_unauthorized (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = da992886-424e-41fe-ae8d-cefa7719ea34 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
118622025-04-30T06:44:11.212ZTRACtest_unauthorized (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
118632025-04-30T06:44:11.213ZDEBGtest_unauthorized (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
118642025-04-30T06:44:11.213ZTRACtest_unauthorized (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
118652025-04-30T06:44:11.213ZTRACtest_unauthorized (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
118662025-04-30T06:44:11.215ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7c216cf2-11a7-4e1d-9e74-35f0b8ac49c1", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
118672025-04-30T06:44:11.215ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
118682025-04-30T06:44:11.216ZINFOtest_unauthorized (dropshot_internal): request completed latency_us = 10824 local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = da992886-424e-41fe-ae8d-cefa7719ea34 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
118692025-04-30T06:44:11.216ZDEBGtest_unauthorized (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "da992886-424e-41fe-ae8d-cefa7719ea34", "content-length": "29", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
118702025-04-30T06:44:11.216ZTRACtest_unauthorized (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 8d752adb-08b9-405f-a11b-66019bf447b5
118712025-04-30T06:44:11.216ZTRACtest_unauthorized (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
118722025-04-30T06:44:11.216ZINFOtest_unauthorized (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
118732025-04-30T06:44:11.219ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7083c23c-077c-480d-9feb-a3739c42937b", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
118742025-04-30T06:44:11.219ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
118752025-04-30T06:44:11.221ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3383e4b1-c0a8-4904-8d3c-af8c7fcf018f", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
118762025-04-30T06:44:11.221ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
118772025-04-30T06:44:11.225ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b70f1bdb-e056-4d0a-9913-0ae30208f05a", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
118782025-04-30T06:44:11.226ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
118792025-04-30T06:44:11.235ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ac1d6759-d4fc-4009-a962-77c69fe41ec6", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
118802025-04-30T06:44:11.235ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
118812025-04-30T06:44:11.238ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c08e466f-2238-497c-9ae2-6602d188cfc6", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
118822025-04-30T06:44:11.238ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
118832025-04-30T06:44:11.239ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e17bc1f9-1679-4d7d-9688-cf63201ad2ee", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
118842025-04-30T06:44:11.239ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
118852025-04-30T06:44:11.247ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "dfd08529-e5ae-4bdf-9e12-59ebdbb953cf", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
118862025-04-30T06:44:11.247ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
118872025-04-30T06:44:11.248ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
118882025-04-30T06:44:11.251ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6c49b490-3bb2-4247-8c1c-785ece18c8f5", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
118892025-04-30T06:44:11.252ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
118902025-04-30T06:44:11.256ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f5b4ecd1-e5ee-4c2e-bf41-5896a752b5bf", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
118912025-04-30T06:44:11.256ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
118922025-04-30T06:44:11.258ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "021e8ae5-eb16-458e-8fbc-4a6b96e32170", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
118932025-04-30T06:44:11.258ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
118942025-04-30T06:44:11.264ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "27280713-f7f6-469a-be50-2aca7f744ba8", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
118952025-04-30T06:44:11.264ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
118962025-04-30T06:44:11.270ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "131dc16a-3246-45e7-ac39-280a511db09f", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
118972025-04-30T06:44:11.270ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
118982025-04-30T06:44:11.275ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1ea093ac-eecd-40dc-bd49-b22bd1a10b06", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
118992025-04-30T06:44:11.275ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
119002025-04-30T06:44:11.277ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a49b05f5-e331-41b1-9191-9b549f877dff", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
119012025-04-30T06:44:11.277ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
119022025-04-30T06:44:11.281ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d3d5d44b-caca-42cc-bab9-34806eace364", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
119032025-04-30T06:44:11.281ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
119042025-04-30T06:44:11.283ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4e2a557c-34cf-4f82-a4a5-6d0116203377", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
119052025-04-30T06:44:11.283ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
119062025-04-30T06:44:11.285ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8b36e899-f694-4876-8e59-271a64640ca5", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
119072025-04-30T06:44:11.287ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
119082025-04-30T06:44:11.290ZWARNtest_unauthorized (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
119092025-04-30T06:44:11.292ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6449befc-e60b-4981-aed9-6050c16417ee", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
119102025-04-30T06:44:11.292ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
119112025-04-30T06:44:11.294ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "492705e5-d149-4222-8618-bb61d2d258b1", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
119122025-04-30T06:44:11.294ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
119132025-04-30T06:44:11.295ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1749e555-299e-4f38-b4e4-9c7ab9d92c93", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
119142025-04-30T06:44:11.295ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
119152025-04-30T06:44:11.297ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "14602aa1-2508-43ca-92b7-1e87732cdeca", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
119162025-04-30T06:44:11.297ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
119172025-04-30T06:44:11.299ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "606e350e-4c3d-4a64-a814-d71a1a2e523a", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
119182025-04-30T06:44:11.299ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
119192025-04-30T06:44:11.300ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
119202025-04-30T06:44:11.300ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
119212025-04-30T06:44:11.300ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
119222025-04-30T06:44:11.300ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
119232025-04-30T06:44:11.301ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
119242025-04-30T06:44:11.303ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "87a09f23-3a20-47ed-ba15-e5d5f3a3e4e1", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
119252025-04-30T06:44:11.303ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
119262025-04-30T06:44:11.312ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119272025-04-30T06:44:11.321ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
119282025-04-30T06:44:11.321ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
119292025-04-30T06:44:11.321ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
119302025-04-30T06:44:11.321ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
119312025-04-30T06:44:11.324ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b90d3612-ff9a-4596-afa6-3766ff2c18bb", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
119322025-04-30T06:44:11.325ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
119332025-04-30T06:44:11.326ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
119342025-04-30T06:44:11.326ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
119352025-04-30T06:44:11.327ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
119362025-04-30T06:44:11.331ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2122ad1e-f561-4744-9c3d-470c474ec709", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
119372025-04-30T06:44:11.331ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
119382025-04-30T06:44:11.343ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8184c609-bac5-40a4-afeb-d1c560e4f3fa", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
119392025-04-30T06:44:11.344ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
119402025-04-30T06:44:11.347ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7f1b77a6-486f-4bb9-88db-5033c1e89537", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
119412025-04-30T06:44:11.347ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
119422025-04-30T06:44:11.350ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4c46d8c6-761a-4c14-987d-c72d0020e730", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
119432025-04-30T06:44:11.350ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
119442025-04-30T06:44:11.352ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5661dbe2-ca76-463d-8e3b-a56af6a7954d", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
119452025-04-30T06:44:11.352ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
119462025-04-30T06:44:11.355ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "70b05a34-e568-41f5-9e8d-d9d7942f68d5", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
119472025-04-30T06:44:11.355ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
119482025-04-30T06:44:11.357ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "58b1491f-b1ec-4fe9-9838-6f884a0ffc8b", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
119492025-04-30T06:44:11.357ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
119502025-04-30T06:44:11.359ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "53b5c54f-3230-42e5-8773-8f70c86f1467", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
119512025-04-30T06:44:11.359ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
119522025-04-30T06:44:11.362ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f4236086-f5e5-402f-9fa2-4b360211543f", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
119532025-04-30T06:44:11.363ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
119542025-04-30T06:44:11.365ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c45309d4-cab7-4139-b542-05f436038e24", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
119552025-04-30T06:44:11.365ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
119562025-04-30T06:44:11.368ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bc97f013-bc21-4103-b17e-eda2e5c4ac75", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
119572025-04-30T06:44:11.369ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
119582025-04-30T06:44:11.370ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "adabee5e-0a8b-4ee4-b36f-222be0f2c639", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
119592025-04-30T06:44:11.371ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
119602025-04-30T06:44:11.372ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9e6b3779-d14b-4fee-aeaa-2eec6bba8b06", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
119612025-04-30T06:44:11.373ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
119622025-04-30T06:44:11.376ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "16808831-f2a9-4bbb-add8-9e7e57322953", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
119632025-04-30T06:44:11.376ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
119642025-04-30T06:44:11.379ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa2eb10b-5041-46b8-89e5-a4fff57d5936", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
119652025-04-30T06:44:11.379ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
119662025-04-30T06:44:11.380ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "33688329-24ac-480b-8649-602682699585", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
119672025-04-30T06:44:11.381ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
119682025-04-30T06:44:11.382ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "180ede98-8315-478f-a3e3-761e712a3293", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
119692025-04-30T06:44:11.383ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
119702025-04-30T06:44:11.384ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "356ef5c7-9fcb-46ee-b03a-b1da149a5e6b", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
119712025-04-30T06:44:11.385ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
119722025-04-30T06:44:11.385ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
119732025-04-30T06:44:11.385ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
119742025-04-30T06:44:11.385ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
119752025-04-30T06:44:11.386ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
119762025-04-30T06:44:11.389ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119772025-04-30T06:44:11.393ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
119782025-04-30T06:44:11.393ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
119792025-04-30T06:44:11.393ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
119802025-04-30T06:44:11.393ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
119812025-04-30T06:44:11.393ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
119822025-04-30T06:44:11.394ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
119832025-04-30T06:44:11.397ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119842025-04-30T06:44:11.402ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
119852025-04-30T06:44:11.402ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
119862025-04-30T06:44:11.402ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
119872025-04-30T06:44:11.403ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
119882025-04-30T06:44:11.406ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
119892025-04-30T06:44:11.464ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
119902025-04-30T06:44:11.464ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory
119912025-04-30T06:44:11.465ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
119922025-04-30T06:44:11.465ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
119932025-04-30T06:44:11.465ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
119942025-04-30T06:44:11.465ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
119952025-04-30T06:44:11.466ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
119962025-04-30T06:44:11.467ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:56246 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
119972025-04-30T06:44:11.468ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:56246 req_id = 14f99388-ab6b-4e72-84f9-16374319bc23 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
119982025-04-30T06:44:11.469ZINFOtest_unauthorized (dropshot): request completed latency_us = 910 local_addr = [::1]:57786 method = GET remote_addr = [::1]:56246 req_id = 14f99388-ab6b-4e72-84f9-16374319bc23 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
119992025-04-30T06:44:11.470ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = http://[::1]:57786 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:57786/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "14f99388-ab6b-4e72-84f9-16374319bc23", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
120002025-04-30T06:44:11.473ZDEBGtest_unauthorized (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
120012025-04-30T06:44:11.473ZDEBGtest_unauthorized (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
120022025-04-30T06:44:11.473ZDEBGtest_unauthorized (BackgroundTasks): finished collection background_task = inventory_collection
120032025-04-30T06:44:11.476ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
120042025-04-30T06:44:11.476ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
120052025-04-30T06:44:11.477ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
120062025-04-30T06:44:11.477ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
120072025-04-30T06:44:11.479ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
120082025-04-30T06:44:11.480ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120092025-04-30T06:44:11.486ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
120102025-04-30T06:44:11.486ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
120112025-04-30T06:44:11.486ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
120122025-04-30T06:44:11.487ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
120132025-04-30T06:44:11.489ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120142025-04-30T06:44:11.496ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
120152025-04-30T06:44:11.496ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
120162025-04-30T06:44:11.498ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
120172025-04-30T06:44:11.499ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
120182025-04-30T06:44:11.499ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
120192025-04-30T06:44:11.499ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
120202025-04-30T06:44:11.500ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
120212025-04-30T06:44:11.531ZWARNtest_unauthorized (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
120222025-04-30T06:44:11.531ZWARNtest_unauthorized (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
120232025-04-30T06:44:11.531ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 556.781521ms iteration = 42 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
120242025-04-30T06:44:11.531ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 43 reason = Signaled
120252025-04-30T06:44:11.531ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
120262025-04-30T06:44:11.531ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
120272025-04-30T06:44:11.531ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
120282025-04-30T06:44:11.531ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
120292025-04-30T06:44:11.532ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
120302025-04-30T06:44:11.548ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
120312025-04-30T06:44:11.550ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120322025-04-30T06:44:11.556ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
120332025-04-30T06:44:11.556ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
120342025-04-30T06:44:11.556ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
120352025-04-30T06:44:11.558ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
120362025-04-30T06:44:11.562ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1a8b2318-abf8-406d-a389-112ab91a0de8, 0)]
120372025-04-30T06:44:11.562ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
120382025-04-30T06:44:11.567ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: e1 68 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .h..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:55421 req_id = 8623e824-a234-40d6-964d-3983fe34222d
120392025-04-30T06:44:11.567ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 57704,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:55421 req_id = 8623e824-a234-40d6-964d-3983fe34222d
120402025-04-30T06:44:11.567ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
120412025-04-30T06:44:11.567ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
120422025-04-30T06:44:11.567ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:55421 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 8623e824-a234-40d6-964d-3983fe34222d
120432025-04-30T06:44:11.570ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4316, tv_nsec: 368401855 } })
120442025-04-30T06:44:11.570ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
120452025-04-30T06:44:11.572ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: e7 1b 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:61426 req_id = 7d49d4ec-fb60-4c00-81ed-36db53f5bf5f
120462025-04-30T06:44:11.573ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 59163,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:61426 req_id = 7d49d4ec-fb60-4c00-81ed-36db53f5bf5f
120472025-04-30T06:44:11.573ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
120482025-04-30T06:44:11.573ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:61426 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 7d49d4ec-fb60-4c00-81ed-36db53f5bf5f
120492025-04-30T06:44:11.633ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
120502025-04-30T06:44:11.634ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
120512025-04-30T06:44:11.634ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
120522025-04-30T06:44:11.635ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
120532025-04-30T06:44:11.635ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
120542025-04-30T06:44:11.635ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
120552025-04-30T06:44:11.644ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 9e cb 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:60062 req_id = 9efd99ab-d1ce-42ad-8d37-8ac3b611ac9a
120562025-04-30T06:44:11.644ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 40651,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60062 req_id = 9efd99ab-d1ce-42ad-8d37-8ac3b611ac9a
120572025-04-30T06:44:11.644ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
120582025-04-30T06:44:11.644ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 40651, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:60062 req_id = 9efd99ab-d1ce-42ad-8d37-8ac3b611ac9a
120592025-04-30T06:44:11.652ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
120602025-04-30T06:44:11.652ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
120612025-04-30T06:44:11.652ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
120622025-04-30T06:44:11.662ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120632025-04-30T06:44:11.686ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
120642025-04-30T06:44:11.686ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
120652025-04-30T06:44:11.686ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
120662025-04-30T06:44:11.687ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
120672025-04-30T06:44:11.700ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "4d464c35-c022-4db7-82e6-763e78665c87", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
120682025-04-30T06:44:11.700ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
120692025-04-30T06:44:11.701ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
120702025-04-30T06:44:11.701ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
120712025-04-30T06:44:11.702ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
120722025-04-30T06:44:11.706ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "86abfa04-41fb-4467-a92b-dbda49b653ab", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
120732025-04-30T06:44:11.706ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
120742025-04-30T06:44:11.714ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b91a41ba-7bb5-430f-b0d4-549271aab993", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
120752025-04-30T06:44:11.714ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
120762025-04-30T06:44:11.717ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ef74eba5-493c-4194-82af-b69c00736c36", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
120772025-04-30T06:44:11.718ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
120782025-04-30T06:44:11.721ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9b46eb32-92d0-4252-85a7-bac2ebeb6fcb", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
120792025-04-30T06:44:11.721ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
120802025-04-30T06:44:11.723ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "332e0ede-6987-4442-8f13-0d28224e6d58", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
120812025-04-30T06:44:11.723ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
120822025-04-30T06:44:11.726ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "96299f8f-5322-4fc1-9656-8dcbe4d4610b", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
120832025-04-30T06:44:11.727ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
120842025-04-30T06:44:11.730ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8fb70280-c478-461a-ab31-2d23d4d5d692", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
120852025-04-30T06:44:11.730ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
120862025-04-30T06:44:11.731ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "201d4c3b-48aa-4296-8cf1-e37ab9b7f38a", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
120872025-04-30T06:44:11.732ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
120882025-04-30T06:44:11.735ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4f744222-0676-4f87-994b-ef20ab941243", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
120892025-04-30T06:44:11.735ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
120902025-04-30T06:44:11.736ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "afba3f81-c337-4bb3-85e9-06cbf50ee23a", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
120912025-04-30T06:44:11.737ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
120922025-04-30T06:44:11.738ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "92653037-917a-483e-a5ad-b67e925f9c7b", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
120932025-04-30T06:44:11.738ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
120942025-04-30T06:44:11.747ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e12eef51-a571-4490-892f-af3a2d59dd55", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
120952025-04-30T06:44:11.747ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
120962025-04-30T06:44:11.754ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
120972025-04-30T06:44:11.756ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f5e7aec2-4778-4802-b1e0-5eb10c903728", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
120982025-04-30T06:44:11.756ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
120992025-04-30T06:44:11.766ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7bfbeb9d-9abb-4b85-bf53-cd870b0d1e8b", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
121002025-04-30T06:44:11.767ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
121012025-04-30T06:44:11.771ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "230cfa17-857e-4b74-b3cc-0d272850610d", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
121022025-04-30T06:44:11.771ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
121032025-04-30T06:44:11.776ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "df9c811f-951f-45b8-a8e4-593bde6f8c94", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
121042025-04-30T06:44:11.776ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
121052025-04-30T06:44:11.778ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1577afba-175e-4c67-9ad9-60b040a6a5c3", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
121062025-04-30T06:44:11.779ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
121072025-04-30T06:44:11.781ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "84c79027-60c3-4fd6-85f2-89bee7f4fde3", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
121082025-04-30T06:44:11.781ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
121092025-04-30T06:44:11.782ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a916bc87-0cde-4402-b76e-71c565242750", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
121102025-04-30T06:44:11.782ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
121112025-04-30T06:44:11.784ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d8da9a80-114d-462f-8c4b-b3c8ba8b7b1f", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
121122025-04-30T06:44:11.785ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
121132025-04-30T06:44:11.787ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "93f22a3b-af1c-41bf-8d26-4428dac1f0d7", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
121142025-04-30T06:44:11.788ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
121152025-04-30T06:44:11.790ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "09bfe3ab-cb97-4617-beb4-35f2773b5511", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
121162025-04-30T06:44:11.790ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
121172025-04-30T06:44:11.791ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cfd45100-d7f2-4aa5-ba36-27dbe9f987b6", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
121182025-04-30T06:44:11.791ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
121192025-04-30T06:44:11.795ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8ea5602d-d304-4b11-a9e6-01b400fc454c", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
121202025-04-30T06:44:11.795ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
121212025-04-30T06:44:11.798ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ba506c28-3fbf-40c3-890d-d2cb96940c05", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
121222025-04-30T06:44:11.798ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
121232025-04-30T06:44:11.800ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9acf2741-81af-42c5-afda-c9b13a1fcc89", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
121242025-04-30T06:44:11.801ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
121252025-04-30T06:44:11.802ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4a1bf9dd-bf39-4ce5-a603-7e14c62ae4cf", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
121262025-04-30T06:44:11.803ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
121272025-04-30T06:44:11.805ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
121282025-04-30T06:44:11.805ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
121292025-04-30T06:44:11.805ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
121302025-04-30T06:44:11.805ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
121312025-04-30T06:44:11.806ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
121322025-04-30T06:44:11.809ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "89e96b48-854f-4320-9dd7-26150c09db8d", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
121332025-04-30T06:44:11.809ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
121342025-04-30T06:44:11.812ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "66f7cbb3-222b-413d-af93-14772d5e4c4d", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
121352025-04-30T06:44:11.813ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
121362025-04-30T06:44:11.813ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5d8f78a0-fbc1-49e8-bd41-b5919f41ebed", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
121372025-04-30T06:44:11.814ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
121382025-04-30T06:44:11.814ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121392025-04-30T06:44:11.830ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
121402025-04-30T06:44:11.830ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
121412025-04-30T06:44:11.830ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
121422025-04-30T06:44:11.839ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
121432025-04-30T06:44:11.843ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
121442025-04-30T06:44:11.843ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
121452025-04-30T06:44:11.843ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
121462025-04-30T06:44:11.844ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0d67d54a-939f-43fd-9a6c-393529d19319", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
121472025-04-30T06:44:11.844ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
121482025-04-30T06:44:11.847ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a1f09ea7-d67c-4113-9322-bffea31eae8f", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
121492025-04-30T06:44:11.847ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
121502025-04-30T06:44:11.848ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f32933e7-f083-441f-bc03-a5d9d4c3a665", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
121512025-04-30T06:44:11.849ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
121522025-04-30T06:44:11.850ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a3a97802-f463-4530-970a-947e3da042ca", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
121532025-04-30T06:44:11.850ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
121542025-04-30T06:44:11.862ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2714cb51-0ab8-4cb0-a0dd-3e316d617ada", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
121552025-04-30T06:44:11.863ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
121562025-04-30T06:44:11.867ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6c206631-8023-4ca7-b816-737109daa374", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
121572025-04-30T06:44:11.867ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
121582025-04-30T06:44:11.870ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e5f71e4e-0f49-4dbe-93cf-b6b4f6b3fdb8", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
121592025-04-30T06:44:11.871ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
121602025-04-30T06:44:11.876ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6427f4c2-9758-416b-bd93-527e62717f9b", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
121612025-04-30T06:44:11.877ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
121622025-04-30T06:44:11.881ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e6565a9f-fcb1-49ed-a798-0bb3bf2d524f", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
121632025-04-30T06:44:11.882ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
121642025-04-30T06:44:11.888ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
121652025-04-30T06:44:11.891ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4603294f-1379-4ef0-8dbe-5add8049d62a", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
121662025-04-30T06:44:11.892ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
121672025-04-30T06:44:11.893ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "86aef943-0dbb-4344-99cb-40d3ca273445", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
121682025-04-30T06:44:11.894ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
121692025-04-30T06:44:11.895ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7eae2ea5-3451-4dfc-83bf-22cb7937e3e1", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
121702025-04-30T06:44:11.896ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
121712025-04-30T06:44:11.897ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "13ffaef3-81e1-4f4b-a81d-838c4ecee536", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
121722025-04-30T06:44:11.898ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
121732025-04-30T06:44:11.899ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3cf6799d-ebb3-422b-a13e-8792d50ea965", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
121742025-04-30T06:44:11.899ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
121752025-04-30T06:44:11.900ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
121762025-04-30T06:44:11.900ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
121772025-04-30T06:44:11.900ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
121782025-04-30T06:44:11.903ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
121792025-04-30T06:44:11.909ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121802025-04-30T06:44:11.914ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
121812025-04-30T06:44:11.914ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
121822025-04-30T06:44:11.914ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
121832025-04-30T06:44:11.914ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
121842025-04-30T06:44:11.914ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
121852025-04-30T06:44:11.914ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
121862025-04-30T06:44:11.919ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121872025-04-30T06:44:11.924ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
121882025-04-30T06:44:11.924ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
121892025-04-30T06:44:11.924ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
121902025-04-30T06:44:11.925ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
121912025-04-30T06:44:11.993ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
121922025-04-30T06:44:11.993ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory
121932025-04-30T06:44:11.993ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
121942025-04-30T06:44:11.994ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
121952025-04-30T06:44:11.994ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
121962025-04-30T06:44:11.994ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
121972025-04-30T06:44:11.995ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
121982025-04-30T06:44:11.995ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:59491 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
121992025-04-30T06:44:11.995ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:59491 req_id = 73aaedfa-667a-4c0b-98e0-68089e3bb34e sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
122002025-04-30T06:44:11.997ZINFOtest_unauthorized (dropshot): request completed latency_us = 1603 local_addr = [::1]:57786 method = GET remote_addr = [::1]:59491 req_id = 73aaedfa-667a-4c0b-98e0-68089e3bb34e response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
122012025-04-30T06:44:11.998ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = http://[::1]:57786 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:57786/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "73aaedfa-667a-4c0b-98e0-68089e3bb34e", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
122022025-04-30T06:44:11.999ZDEBGtest_unauthorized (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
122032025-04-30T06:44:11.999ZDEBGtest_unauthorized (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
122042025-04-30T06:44:11.999ZDEBGtest_unauthorized (BackgroundTasks): finished collection background_task = inventory_collection
122052025-04-30T06:44:12.004ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
122062025-04-30T06:44:12.004ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
122072025-04-30T06:44:12.004ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
122082025-04-30T06:44:12.004ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
122092025-04-30T06:44:12.005ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
122102025-04-30T06:44:12.008ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122112025-04-30T06:44:12.017ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
122122025-04-30T06:44:12.017ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
122132025-04-30T06:44:12.017ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
122142025-04-30T06:44:12.017ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
122152025-04-30T06:44:12.019ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122162025-04-30T06:44:12.025ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
122172025-04-30T06:44:12.025ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
122182025-04-30T06:44:12.025ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
122192025-04-30T06:44:12.026ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
122202025-04-30T06:44:12.026ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
122212025-04-30T06:44:12.026ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
122222025-04-30T06:44:12.027ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
122232025-04-30T06:44:12.071ZWARNtest_unauthorized (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
122242025-04-30T06:44:12.071ZWARNtest_unauthorized (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
122252025-04-30T06:44:12.071ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 540.143842ms iteration = 43 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
122262025-04-30T06:44:12.071ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 44 reason = Signaled
122272025-04-30T06:44:12.071ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
122282025-04-30T06:44:12.072ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
122292025-04-30T06:44:12.072ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
122302025-04-30T06:44:12.072ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
122312025-04-30T06:44:12.072ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
122322025-04-30T06:44:12.075ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122332025-04-30T06:44:12.079ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
122342025-04-30T06:44:12.079ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
122352025-04-30T06:44:12.079ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
122362025-04-30T06:44:12.080ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
122372025-04-30T06:44:12.085ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1a8b2318-abf8-406d-a389-112ab91a0de8, 0)]
122382025-04-30T06:44:12.085ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
122392025-04-30T06:44:12.086ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: e0 9f 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:57713 req_id = d9d9c0a6-a2d7-43c2-b66f-ed1f5be94d05
122402025-04-30T06:44:12.086ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 57503,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:57713 req_id = d9d9c0a6-a2d7-43c2-b66f-ed1f5be94d05
122412025-04-30T06:44:12.086ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
122422025-04-30T06:44:12.088ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
122432025-04-30T06:44:12.088ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:57713 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = d9d9c0a6-a2d7-43c2-b66f-ed1f5be94d05
122442025-04-30T06:44:12.094ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4316, tv_nsec: 893638435 } })
122452025-04-30T06:44:12.096ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
122462025-04-30T06:44:12.096ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
122472025-04-30T06:44:12.097ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: f0 df 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:50455 req_id = 7a652750-aae9-4576-aebb-1d3cd7d146da
122482025-04-30T06:44:12.097ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 61663,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:50455 req_id = 7a652750-aae9-4576-aebb-1d3cd7d146da
122492025-04-30T06:44:12.098ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
122502025-04-30T06:44:12.098ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:50455 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 7a652750-aae9-4576-aebb-1d3cd7d146da
122512025-04-30T06:44:12.172ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
122522025-04-30T06:44:12.172ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
122532025-04-30T06:44:12.173ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
122542025-04-30T06:44:12.173ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
122552025-04-30T06:44:12.173ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
122562025-04-30T06:44:12.174ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
122572025-04-30T06:44:12.174ZDEBGtest_unauthorized (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
122582025-04-30T06:44:12.175ZDEBGtest_unauthorized (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
122592025-04-30T06:44:12.175ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: db 60 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c .`..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:51159 req_id = 0b93a2fb-4188-4562-9c78-0c57410ee02e
122602025-04-30T06:44:12.175ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 56160,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51159 req_id = 0b93a2fb-4188-4562-9c78-0c57410ee02e
122612025-04-30T06:44:12.176ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
122622025-04-30T06:44:12.176ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 56160, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:51159 req_id = 0b93a2fb-4188-4562-9c78-0c57410ee02e
122632025-04-30T06:44:12.176ZDEBGtest_unauthorized (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a
122642025-04-30T06:44:12.176ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
122652025-04-30T06:44:12.178ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
122662025-04-30T06:44:12.180ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
122672025-04-30T06:44:12.180ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
122682025-04-30T06:44:12.182ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122692025-04-30T06:44:12.204ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
122702025-04-30T06:44:12.207ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
122712025-04-30T06:44:12.207ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
122722025-04-30T06:44:12.208ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
122732025-04-30T06:44:12.214ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 2 table_name = fields_string
122742025-04-30T06:44:12.214ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
122752025-04-30T06:44:12.214ZDEBGtest_unauthorized (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
122762025-04-30T06:44:12.214ZDEBGtest_unauthorized (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
122772025-04-30T06:44:12.216ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
122782025-04-30T06:44:12.216ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
122792025-04-30T06:44:12.217ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
122802025-04-30T06:44:12.233ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 1 table_name = measurements_i64
122812025-04-30T06:44:12.233ZTRACtest_unauthorized (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
122822025-04-30T06:44:12.236ZTRACtest_unauthorized (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
122832025-04-30T06:44:12.237ZDEBGtest_unauthorized (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
122842025-04-30T06:44:12.237ZTRACtest_unauthorized (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
122852025-04-30T06:44:12.237ZTRACtest_unauthorized (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
122862025-04-30T06:44:12.238ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0b67d1a-803d-4968-b90c-f1945af94531", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
122872025-04-30T06:44:12.240ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
122882025-04-30T06:44:12.244ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5ca9f000-499c-48f8-be8d-ddcbc1d3249e", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
122892025-04-30T06:44:12.244ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
122902025-04-30T06:44:12.246ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "15d31d98-4313-403e-83db-b4b8a240485a", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
122912025-04-30T06:44:12.247ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
122922025-04-30T06:44:12.252ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e539e9b4-733f-4efc-b8f0-3f9569730dfd", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
122932025-04-30T06:44:12.253ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
122942025-04-30T06:44:12.256ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cbd54b9e-379a-4c89-a486-05dc0f7ee5d6", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
122952025-04-30T06:44:12.257ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
122962025-04-30T06:44:12.259ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "84aa6e62-13fc-48bb-80ab-e9330cc84ea8", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
122972025-04-30T06:44:12.259ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
122982025-04-30T06:44:12.264ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6c6d98cf-dc19-46b5-904f-cccd73eda57d", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
122992025-04-30T06:44:12.264ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
123002025-04-30T06:44:12.270ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9664d6f9-579b-40e7-acd9-9db42bff52da", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
123012025-04-30T06:44:12.270ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
123022025-04-30T06:44:12.272ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "38570f04-02fb-4c2a-994f-cc44cc03a233", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
123032025-04-30T06:44:12.272ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
123042025-04-30T06:44:12.277ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2dfc18b7-5ec3-4cac-89db-245a74c16146", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
123052025-04-30T06:44:12.277ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
123062025-04-30T06:44:12.278ZWARNtest_unauthorized (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
123072025-04-30T06:44:12.278ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
123082025-04-30T06:44:12.281ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ede48c81-6632-4d07-8cda-644e9c47dcbf", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
123092025-04-30T06:44:12.281ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
123102025-04-30T06:44:12.283ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2e0538da-80b9-4602-a200-4c975937eb4b", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
123112025-04-30T06:44:12.283ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
123122025-04-30T06:44:12.285ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "92c86e29-785a-4071-8418-c68005842995", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
123132025-04-30T06:44:12.285ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
123142025-04-30T06:44:12.288ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2f6488e8-01e0-427a-a5fc-43afd9ea5d6a", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
123152025-04-30T06:44:12.288ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
123162025-04-30T06:44:12.306ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b1d53441-6df2-4f4b-8a42-b4b8363b761d", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
123172025-04-30T06:44:12.306ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
123182025-04-30T06:44:12.308ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c5bd7ce8-3981-4a86-a0e8-b871462738cf", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
123192025-04-30T06:44:12.308ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
123202025-04-30T06:44:12.313ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8f8c3483-6925-43ca-92ac-8dec4847ba31", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
123212025-04-30T06:44:12.313ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
123222025-04-30T06:44:12.317ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7263bb69-2a93-4098-8364-a7923593330d", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
123232025-04-30T06:44:12.318ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
123242025-04-30T06:44:12.322ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0e30815b-e32c-4f53-82b3-0da945e1f834", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
123252025-04-30T06:44:12.322ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
123262025-04-30T06:44:12.323ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "489ed4a3-e171-4959-a4ed-d6d71a1050de", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
123272025-04-30T06:44:12.332ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
123282025-04-30T06:44:12.332ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
123292025-04-30T06:44:12.332ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
123302025-04-30T06:44:12.332ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
123312025-04-30T06:44:12.332ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
123322025-04-30T06:44:12.333ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
123332025-04-30T06:44:12.337ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a754f25b-97bf-4625-9f9c-ca9d32e42b91", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
123342025-04-30T06:44:12.337ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
123352025-04-30T06:44:12.339ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123362025-04-30T06:44:12.346ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
123372025-04-30T06:44:12.346ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
123382025-04-30T06:44:12.346ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
123392025-04-30T06:44:12.346ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
123402025-04-30T06:44:12.348ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "48f4b319-7c44-4dad-8dce-9210284e139b", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
123412025-04-30T06:44:12.348ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
123422025-04-30T06:44:12.349ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
123432025-04-30T06:44:12.349ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
123442025-04-30T06:44:12.350ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
123452025-04-30T06:44:12.351ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4bee521b-eec3-4e8b-be82-a39160b1b4cd", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
123462025-04-30T06:44:12.351ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
123472025-04-30T06:44:12.354ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e029f6aa-67dc-4a58-9e3c-ecf307e9cabc", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
123482025-04-30T06:44:12.355ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
123492025-04-30T06:44:12.357ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "df0b3222-b6d5-4e7e-b144-41c9194b7e21", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
123502025-04-30T06:44:12.358ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
123512025-04-30T06:44:12.362ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8ee4c035-e576-4a17-aeb9-44be81157595", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
123522025-04-30T06:44:12.362ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
123532025-04-30T06:44:12.365ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "dafd4121-0f59-4330-b056-ec6ee903c864", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
123542025-04-30T06:44:12.365ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
123552025-04-30T06:44:12.369ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4bd15121-2b05-4fad-957f-0e8bcfc2741b", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
123562025-04-30T06:44:12.369ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
123572025-04-30T06:44:12.372ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bc1b4c72-2873-47d9-ab56-991c1e496387", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
123582025-04-30T06:44:12.372ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
123592025-04-30T06:44:12.378ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ea3b0544-3bed-4083-8a47-21a6aa7b92db", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
123602025-04-30T06:44:12.378ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
123612025-04-30T06:44:12.379ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
123622025-04-30T06:44:12.381ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3ab46d80-9cdd-4d8c-8505-de485f2230a7", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
123632025-04-30T06:44:12.381ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
123642025-04-30T06:44:12.382ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fdd60085-477c-4854-bc54-a607296c285e", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
123652025-04-30T06:44:12.382ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
123662025-04-30T06:44:12.383ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "86390386-19ae-4e15-9bbb-f18079e04701", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
123672025-04-30T06:44:12.384ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
123682025-04-30T06:44:12.385ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5b4e9dc8-2489-4774-81d2-99528ec5c18d", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
123692025-04-30T06:44:12.385ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
123702025-04-30T06:44:12.386ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9137548a-56c3-464b-86e9-6a06694c5c55", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
123712025-04-30T06:44:12.387ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
123722025-04-30T06:44:12.391ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d6179fb7-3305-4dcb-b74e-b7e976715407", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
123732025-04-30T06:44:12.391ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
123742025-04-30T06:44:12.395ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c9ee223b-b421-45a2-87e4-382e20824712", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
123752025-04-30T06:44:12.395ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
123762025-04-30T06:44:12.398ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ec1cea72-513c-433b-89ce-6b8c7a26fff9", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
123772025-04-30T06:44:12.398ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
123782025-04-30T06:44:12.401ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "efcc8433-d9ab-483a-a542-316c092f55ef", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
123792025-04-30T06:44:12.401ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
123802025-04-30T06:44:12.405ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c3454002-bd92-4a25-9dbb-fb3197f93ca9", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
123812025-04-30T06:44:12.405ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
123822025-04-30T06:44:12.407ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4e7c1624-5e47-471b-a5ef-487c779538f7", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
123832025-04-30T06:44:12.407ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
123842025-04-30T06:44:12.408ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4ec78092-f4b5-4862-8622-05f024942714", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
123852025-04-30T06:44:12.408ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
123862025-04-30T06:44:12.416ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "dbae041b-1eee-410d-b414-fd484a6c6752", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
123872025-04-30T06:44:12.418ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
123882025-04-30T06:44:12.420ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d9609fbd-3368-4516-abaa-622df7b330c4", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
123892025-04-30T06:44:12.420ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
123902025-04-30T06:44:12.423ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6dfb7864-a432-434e-bd0c-e52ff768bb79", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
123912025-04-30T06:44:12.424ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
123922025-04-30T06:44:12.424ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
123932025-04-30T06:44:12.424ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
123942025-04-30T06:44:12.424ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
123952025-04-30T06:44:12.429ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
123962025-04-30T06:44:12.432ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
123972025-04-30T06:44:12.432ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
123982025-04-30T06:44:12.432ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
123992025-04-30T06:44:12.432ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
124002025-04-30T06:44:12.433ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
124012025-04-30T06:44:12.435ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124022025-04-30T06:44:12.440ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
124032025-04-30T06:44:12.441ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
124042025-04-30T06:44:12.441ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
124052025-04-30T06:44:12.441ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
124062025-04-30T06:44:12.441ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
124072025-04-30T06:44:12.441ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
124082025-04-30T06:44:12.451ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124092025-04-30T06:44:12.458ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
124102025-04-30T06:44:12.458ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
124112025-04-30T06:44:12.458ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
124122025-04-30T06:44:12.458ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
124132025-04-30T06:44:12.464ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124142025-04-30T06:44:12.470ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
124152025-04-30T06:44:12.471ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
124162025-04-30T06:44:12.471ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
124172025-04-30T06:44:12.472ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
124182025-04-30T06:44:12.477ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
124192025-04-30T06:44:12.477ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
124202025-04-30T06:44:12.477ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
124212025-04-30T06:44:12.533ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
124222025-04-30T06:44:12.533ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory
124232025-04-30T06:44:12.533ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:60238 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
124242025-04-30T06:44:12.534ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:60238 req_id = 9a1cd85f-7464-4b9a-b09d-206225faa0a7 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
124252025-04-30T06:44:12.534ZINFOtest_unauthorized (dropshot): request completed latency_us = 612 local_addr = [::1]:57786 method = GET remote_addr = [::1]:60238 req_id = 9a1cd85f-7464-4b9a-b09d-206225faa0a7 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
124262025-04-30T06:44:12.535ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = http://[::1]:57786 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:57786/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "9a1cd85f-7464-4b9a-b09d-206225faa0a7", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
124272025-04-30T06:44:12.536ZDEBGtest_unauthorized (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
124282025-04-30T06:44:12.536ZDEBGtest_unauthorized (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
124292025-04-30T06:44:12.537ZDEBGtest_unauthorized (BackgroundTasks): finished collection background_task = inventory_collection
124302025-04-30T06:44:12.539ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
124312025-04-30T06:44:12.539ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
124322025-04-30T06:44:12.539ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
124332025-04-30T06:44:12.539ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
124342025-04-30T06:44:12.540ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
124352025-04-30T06:44:12.543ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124362025-04-30T06:44:12.547ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
124372025-04-30T06:44:12.547ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
124382025-04-30T06:44:12.547ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
124392025-04-30T06:44:12.548ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
124402025-04-30T06:44:12.627ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
124412025-04-30T06:44:12.636ZWARNtest_unauthorized (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
124422025-04-30T06:44:12.636ZWARNtest_unauthorized (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
124432025-04-30T06:44:12.636ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 564.804477ms iteration = 44 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
124442025-04-30T06:44:12.636ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 45 reason = Signaled
124452025-04-30T06:44:12.637ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
124462025-04-30T06:44:12.637ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
124472025-04-30T06:44:12.637ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
124482025-04-30T06:44:12.637ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
124492025-04-30T06:44:12.638ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
124502025-04-30T06:44:12.642ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124512025-04-30T06:44:12.651ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
124522025-04-30T06:44:12.651ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
124532025-04-30T06:44:12.651ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
124542025-04-30T06:44:12.652ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
124552025-04-30T06:44:12.656ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1a8b2318-abf8-406d-a389-112ab91a0de8, 0)]
124562025-04-30T06:44:12.656ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
124572025-04-30T06:44:12.659ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 8f 41 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .A..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:54514 req_id = f1e5cc66-3585-493c-8b8e-485d3028ef53
124582025-04-30T06:44:12.659ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 36673,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54514 req_id = f1e5cc66-3585-493c-8b8e-485d3028ef53
124592025-04-30T06:44:12.660ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
124602025-04-30T06:44:12.661ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
124612025-04-30T06:44:12.662ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:54514 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = f1e5cc66-3585-493c-8b8e-485d3028ef53
124622025-04-30T06:44:12.663ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4317, tv_nsec: 461795170 } })
124632025-04-30T06:44:12.663ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
124642025-04-30T06:44:12.664ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: b7 7d 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .}..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:49821 req_id = bd74753f-3075-4324-9c80-884db46b26b8
124652025-04-30T06:44:12.665ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 46973,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:49821 req_id = bd74753f-3075-4324-9c80-884db46b26b8
124662025-04-30T06:44:12.665ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
124672025-04-30T06:44:12.665ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:49821 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = bd74753f-3075-4324-9c80-884db46b26b8
124682025-04-30T06:44:12.728ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
124692025-04-30T06:44:12.728ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
124702025-04-30T06:44:12.728ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
124712025-04-30T06:44:12.728ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
124722025-04-30T06:44:12.728ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
124732025-04-30T06:44:12.730ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
124742025-04-30T06:44:12.730ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 79 cb 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c y............_cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:63489 req_id = d61021db-1ec5-4435-b6fb-cd1352c2f77c
124752025-04-30T06:44:12.730ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 31179,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63489 req_id = d61021db-1ec5-4435-b6fb-cd1352c2f77c
124762025-04-30T06:44:12.731ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
124772025-04-30T06:44:12.731ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 31179, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:63489 req_id = d61021db-1ec5-4435-b6fb-cd1352c2f77c
124782025-04-30T06:44:12.731ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
124792025-04-30T06:44:12.731ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
124802025-04-30T06:44:12.731ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
124812025-04-30T06:44:12.733ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "ee56047a-6e74-4cbb-a5ab-edd5ff576811", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
124822025-04-30T06:44:12.733ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
124832025-04-30T06:44:12.733ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124842025-04-30T06:44:12.738ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
124852025-04-30T06:44:12.738ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
124862025-04-30T06:44:12.738ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
124872025-04-30T06:44:12.739ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
124882025-04-30T06:44:12.741ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "da32d420-6864-416c-8f57-575a2a6084bd", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
124892025-04-30T06:44:12.741ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
124902025-04-30T06:44:12.742ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
124912025-04-30T06:44:12.742ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
124922025-04-30T06:44:12.742ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
124932025-04-30T06:44:12.757ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b2a6316e-4c99-47fd-aa51-e0d22daa1b29", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
124942025-04-30T06:44:12.758ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
124952025-04-30T06:44:12.767ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ec705ae2-06ed-4ad9-9fb2-31a1305e4672", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
124962025-04-30T06:44:12.767ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
124972025-04-30T06:44:12.771ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ec0b8b33-79e3-46f6-8958-82d4e74a71d5", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
124982025-04-30T06:44:12.772ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
124992025-04-30T06:44:12.774ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f8285af5-140b-4e24-a537-32002e564701", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
125002025-04-30T06:44:12.774ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
125012025-04-30T06:44:12.776ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "892c2aff-b4d7-4ee6-b585-5e298b0847d7", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
125022025-04-30T06:44:12.776ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
125032025-04-30T06:44:12.778ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9b8f0b58-5a3b-43a7-98dd-3d8a5faff256", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
125042025-04-30T06:44:12.778ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
125052025-04-30T06:44:12.779ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6aa261c5-3727-477c-a277-7f6d332843a9", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
125062025-04-30T06:44:12.779ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
125072025-04-30T06:44:12.784ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b29a96aa-d62e-4f26-a7bf-2d74e3d2fcd1", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
125082025-04-30T06:44:12.785ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
125092025-04-30T06:44:12.785ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
125102025-04-30T06:44:12.789ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9263fc69-4e9b-4d3e-a396-0ab9bb56bb03", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
125112025-04-30T06:44:12.789ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
125122025-04-30T06:44:12.790ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "83d3a4cd-81cd-4156-b515-84023c63b942", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
125132025-04-30T06:44:12.790ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
125142025-04-30T06:44:12.793ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3981e667-f21f-4ed1-8261-13a2b7705c53", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
125152025-04-30T06:44:12.793ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
125162025-04-30T06:44:12.799ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e7425b79-98cd-4362-b1b3-0cfda6b26a12", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
125172025-04-30T06:44:12.799ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
125182025-04-30T06:44:12.801ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6bf70519-9a7c-4550-a61b-bcf0c626e789", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
125192025-04-30T06:44:12.802ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
125202025-04-30T06:44:12.806ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2e791646-3783-429c-aa9b-f6caffbd584e", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
125212025-04-30T06:44:12.806ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
125222025-04-30T06:44:12.819ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "51a305e3-5256-4574-b40a-5ea258fad794", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
125232025-04-30T06:44:12.819ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
125242025-04-30T06:44:12.824ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4681b004-b72c-4a64-b12f-2b3834be0fe0", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
125252025-04-30T06:44:12.824ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
125262025-04-30T06:44:12.827ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "af8f995d-61ce-4723-9394-7b0579b554b2", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
125272025-04-30T06:44:12.827ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
125282025-04-30T06:44:12.828ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9861a6dc-8817-4594-87c6-7962bb47b04c", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
125292025-04-30T06:44:12.829ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
125302025-04-30T06:44:12.831ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d1694f4c-3a37-46e6-bc5e-d44b086f44a1", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
125312025-04-30T06:44:12.831ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
125322025-04-30T06:44:12.833ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e1470eb3-2224-4729-908c-8f034b20579c", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
125332025-04-30T06:44:12.833ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
125342025-04-30T06:44:12.834ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "77c5e49d-843f-4763-8f52-25a4b3cf00f7", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
125352025-04-30T06:44:12.835ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
125362025-04-30T06:44:12.837ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "09ca6b90-99ce-4ad6-ae6e-5b3d3f7d198b", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
125372025-04-30T06:44:12.837ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
125382025-04-30T06:44:12.838ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
125392025-04-30T06:44:12.838ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
125402025-04-30T06:44:12.838ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
125412025-04-30T06:44:12.838ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
125422025-04-30T06:44:12.841ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
125432025-04-30T06:44:12.844ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125442025-04-30T06:44:12.851ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
125452025-04-30T06:44:12.851ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
125462025-04-30T06:44:12.851ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
125472025-04-30T06:44:12.851ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
125482025-04-30T06:44:12.853ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "13d5ed08-329e-4058-aa2a-7174e5d1d24b", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
125492025-04-30T06:44:12.853ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
125502025-04-30T06:44:12.855ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
125512025-04-30T06:44:12.855ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
125522025-04-30T06:44:12.856ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
125532025-04-30T06:44:12.859ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "17114b73-4b97-4949-92ad-27f18d421c8f", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
125542025-04-30T06:44:12.859ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
125552025-04-30T06:44:12.862ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "88ef230a-bf99-4331-8d12-1fbf70bd38f3", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
125562025-04-30T06:44:12.862ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
125572025-04-30T06:44:12.866ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "77aaa405-1eaf-4e33-8139-bb88163b55ea", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
125582025-04-30T06:44:12.867ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
125592025-04-30T06:44:12.871ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "864b0064-4e4d-4c76-9d46-2ee438f53a2c", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
125602025-04-30T06:44:12.872ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
125612025-04-30T06:44:12.875ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "afd5a0a2-92a2-4041-91d2-d1354297a003", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
125622025-04-30T06:44:12.876ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
125632025-04-30T06:44:12.878ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "15e0e6ce-9429-4992-a0b4-4228a5f55f76", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
125642025-04-30T06:44:12.879ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
125652025-04-30T06:44:12.881ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7d2ff1dc-decd-43cc-8a04-60c23b5ebf0f", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
125662025-04-30T06:44:12.882ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
125672025-04-30T06:44:12.884ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "740382ad-3673-49b8-824e-47ea052561ce", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
125682025-04-30T06:44:12.885ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
125692025-04-30T06:44:12.886ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
125702025-04-30T06:44:12.887ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc81ec72-d24d-4369-85b1-f8ba8293ab29", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
125712025-04-30T06:44:12.887ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
125722025-04-30T06:44:12.890ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "13440c3c-11c2-4ed8-9712-05d388539a4c", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
125732025-04-30T06:44:12.891ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
125742025-04-30T06:44:12.895ZTRACtest_unauthorized (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
125752025-04-30T06:44:12.896ZTRACtest_unauthorized (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: dc df 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 ............._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:60364 req_id = 82d4a130-24c8-4ca6-80fc-028c83cf5162
125762025-04-30T06:44:12.896ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 56543,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60364 req_id = 82d4a130-24c8-4ca6-80fc-028c83cf5162
125772025-04-30T06:44:12.896ZDEBGtest_unauthorized (store): query key key = _dendrite._tcp
125782025-04-30T06:44:12.897ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
125792025-04-30T06:44:12.897ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:60364 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 44264, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 82d4a130-24c8-4ca6-80fc-028c83cf5162
125802025-04-30T06:44:12.897ZDEBGtest_unauthorized (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 44264, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4317, tv_nsec: 696622528 } })
125812025-04-30T06:44:12.897ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
125822025-04-30T06:44:12.898ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3e711a37-7ef7-419f-ad38-cc52d517b9cd", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
125832025-04-30T06:44:12.898ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
125842025-04-30T06:44:12.899ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: d7 1a 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:49349 req_id = 3ac4f85b-215e-48da-8448-3feea17eb0de
125852025-04-30T06:44:12.899ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 55066,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:49349 req_id = 3ac4f85b-215e-48da-8448-3feea17eb0de
125862025-04-30T06:44:12.899ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
125872025-04-30T06:44:12.899ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:49349 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 3ac4f85b-215e-48da-8448-3feea17eb0de
125882025-04-30T06:44:12.899ZINFOtest_unauthorized (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector
125892025-04-30T06:44:12.972ZINFOtest_unauthorized (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector zone_address = ::1
125902025-04-30T06:44:12.972ZDEBGtest_unauthorized (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[::1]:12225/local/switch-id
125912025-04-30T06:44:12.972ZDEBGtest_unauthorized (MgsClient): client response background_task = nat_v4_garbage_collector result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
125922025-04-30T06:44:12.972ZWARNtest_unauthorized (BackgroundTasks): failed to identify switch slot for dendrite background_task = nat_v4_garbage_collector reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
125932025-04-30T06:44:12.975ZWARNtest_unauthorized (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = nat_v4_garbage_collector
125942025-04-30T06:44:12.975ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
125952025-04-30T06:44:12.975ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
125962025-04-30T06:44:12.975ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
125972025-04-30T06:44:12.975ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
125982025-04-30T06:44:12.976ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
125992025-04-30T06:44:12.976ZTRACtest_unauthorized (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
126002025-04-30T06:44:12.977ZTRACtest_unauthorized (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 72 38 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 r8..........._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:65357 req_id = 4af9ab0c-1b60-4585-aec1-5d506d17c7ef
126012025-04-30T06:44:12.977ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 29240,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:65357 req_id = 4af9ab0c-1b60-4585-aec1-5d506d17c7ef
126022025-04-30T06:44:12.978ZDEBGtest_unauthorized (store): query key key = _dendrite._tcp
126032025-04-30T06:44:12.978ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
126042025-04-30T06:44:12.978ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:65357 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 44264, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 4af9ab0c-1b60-4585-aec1-5d506d17c7ef
126052025-04-30T06:44:12.979ZDEBGtest_unauthorized (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 44264, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4317, tv_nsec: 778266739 } })
126062025-04-30T06:44:12.979ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
126072025-04-30T06:44:12.979ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 7d 96 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e }...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:50201 req_id = f2a1bcf4-6af5-408c-bb85-90e441d633ad
126082025-04-30T06:44:12.980ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 32150,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:50201 req_id = f2a1bcf4-6af5-408c-bb85-90e441d633ad
126092025-04-30T06:44:12.980ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
126102025-04-30T06:44:12.980ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:50201 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = f2a1bcf4-6af5-408c-bb85-90e441d633ad
126112025-04-30T06:44:12.980ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126122025-04-30T06:44:12.986ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
126132025-04-30T06:44:12.987ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
126142025-04-30T06:44:12.987ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
126152025-04-30T06:44:12.988ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
126162025-04-30T06:44:12.988ZINFOtest_unauthorized (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager
126172025-04-30T06:44:13.063ZINFOtest_unauthorized (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager zone_address = ::1
126182025-04-30T06:44:13.063ZDEBGtest_unauthorized (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[::1]:12225/local/switch-id
126192025-04-30T06:44:13.065ZDEBGtest_unauthorized (MgsClient): client response background_task = bfd_manager result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
126202025-04-30T06:44:13.065ZWARNtest_unauthorized (BackgroundTasks): failed to identify switch slot for dendrite background_task = bfd_manager reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
126212025-04-30T06:44:13.066ZWARNtest_unauthorized (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = bfd_manager
126222025-04-30T06:44:13.076ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c4b5374c-6145-4303-8026-2ec234ad182b", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
126232025-04-30T06:44:13.076ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
126242025-04-30T06:44:13.077ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
126252025-04-30T06:44:13.077ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
126262025-04-30T06:44:13.079ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
126272025-04-30T06:44:13.085ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d4eaa099-e563-4be6-81d9-77450fc65f5d", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
126282025-04-30T06:44:13.085ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
126292025-04-30T06:44:13.088ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "578289ac-9d77-4c26-b79e-ab26dcaac467", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
126302025-04-30T06:44:13.088ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
126312025-04-30T06:44:13.090ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "560a1fd7-6bef-4c93-850e-c26116591063", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
126322025-04-30T06:44:13.090ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
126332025-04-30T06:44:13.092ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9ae12d7f-55d1-49cb-8b17-94903aaea920", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
126342025-04-30T06:44:13.093ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
126352025-04-30T06:44:13.099ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ac1487d7-6191-4b14-850f-08b20beaf659", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
126362025-04-30T06:44:13.100ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
126372025-04-30T06:44:13.101ZINFOtest_unauthorized (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
126382025-04-30T06:44:13.102ZDEBGtest_unauthorized (oximeter): client request body = None method = GET uri = http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
126392025-04-30T06:44:13.103ZTRACtest_unauthorized (dropshot_internal): incoming request local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = ea8ef360-1b4a-4e99-9986-6786d47c7ccf uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
126402025-04-30T06:44:13.104ZTRACtest_unauthorized (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = ea8ef360-1b4a-4e99-9986-6786d47c7ccf resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
126412025-04-30T06:44:13.104ZDEBGtest_unauthorized (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = ea8ef360-1b4a-4e99-9986-6786d47c7ccf roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
126422025-04-30T06:44:13.105ZDEBGtest_unauthorized (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = ea8ef360-1b4a-4e99-9986-6786d47c7ccf resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
126432025-04-30T06:44:13.107ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c400f664-669d-4a3c-bf10-ce65833db996", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
126442025-04-30T06:44:13.107ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
126452025-04-30T06:44:13.109ZINFOtest_unauthorized (dropshot_internal): request completed latency_us = 6194 local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = ea8ef360-1b4a-4e99-9986-6786d47c7ccf response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
126462025-04-30T06:44:13.111ZDEBGtest_unauthorized (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "ea8ef360-1b4a-4e99-9986-6786d47c7ccf", "content-length": "405", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
126472025-04-30T06:44:13.112ZDEBGtest_unauthorized (oximeter): client request body = None method = GET uri = http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
126482025-04-30T06:44:13.115ZTRACtest_unauthorized (dropshot_internal): incoming request local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = f5e3c16e-beed-4fa6-a23f-c55c572c7b2a uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
126492025-04-30T06:44:13.116ZTRACtest_unauthorized (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = f5e3c16e-beed-4fa6-a23f-c55c572c7b2a resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
126502025-04-30T06:44:13.116ZDEBGtest_unauthorized (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = f5e3c16e-beed-4fa6-a23f-c55c572c7b2a roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
126512025-04-30T06:44:13.120ZDEBGtest_unauthorized (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = f5e3c16e-beed-4fa6-a23f-c55c572c7b2a resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
126522025-04-30T06:44:13.121ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "67e56142-7b50-400d-8207-7440a67a4e96", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
126532025-04-30T06:44:13.121ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
126542025-04-30T06:44:13.122ZINFOtest_unauthorized (dropshot_internal): request completed latency_us = 6963 local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = f5e3c16e-beed-4fa6-a23f-c55c572c7b2a response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
126552025-04-30T06:44:13.123ZDEBGtest_unauthorized (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "f5e3c16e-beed-4fa6-a23f-c55c572c7b2a", "content-length": "29", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
126562025-04-30T06:44:13.123ZTRACtest_unauthorized (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 8d752adb-08b9-405f-a11b-66019bf447b5
126572025-04-30T06:44:13.123ZTRACtest_unauthorized (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
126582025-04-30T06:44:13.123ZINFOtest_unauthorized (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
126592025-04-30T06:44:13.124ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e5172495-1f48-4a86-b587-4d8d2e85b3cc", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
126602025-04-30T06:44:13.124ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
126612025-04-30T06:44:13.124ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
126622025-04-30T06:44:13.124ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
126632025-04-30T06:44:13.124ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
126642025-04-30T06:44:13.125ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
126652025-04-30T06:44:13.131ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126662025-04-30T06:44:13.138ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
126672025-04-30T06:44:13.138ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
126682025-04-30T06:44:13.138ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
126692025-04-30T06:44:13.138ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
126702025-04-30T06:44:13.138ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
126712025-04-30T06:44:13.139ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
126722025-04-30T06:44:13.144ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126732025-04-30T06:44:13.154ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
126742025-04-30T06:44:13.154ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
126752025-04-30T06:44:13.154ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
126762025-04-30T06:44:13.157ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
126772025-04-30T06:44:13.165ZTRACtest_unauthorized (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
126782025-04-30T06:44:13.166ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
126792025-04-30T06:44:13.167ZTRACtest_unauthorized (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 84 40 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 .@..........._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:55028 req_id = d8d58896-82a8-4474-b4b4-4ff36718e1d3
126802025-04-30T06:44:13.167ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 33856,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:55028 req_id = d8d58896-82a8-4474-b4b4-4ff36718e1d3
126812025-04-30T06:44:13.167ZDEBGtest_unauthorized (store): query key key = _dendrite._tcp
126822025-04-30T06:44:13.167ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
126832025-04-30T06:44:13.167ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:55028 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 44264, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = d8d58896-82a8-4474-b4b4-4ff36718e1d3
126842025-04-30T06:44:13.168ZDEBGtest_unauthorized (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 44264, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4317, tv_nsec: 967262691 } })
126852025-04-30T06:44:13.168ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
126862025-04-30T06:44:13.168ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 87 91 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:65234 req_id = b4689c0b-b2d8-4f31-94fa-ee8bdf6967e7
126872025-04-30T06:44:13.168ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 34705,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:65234 req_id = b4689c0b-b2d8-4f31-94fa-ee8bdf6967e7
126882025-04-30T06:44:13.169ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
126892025-04-30T06:44:13.169ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:65234 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = b4689c0b-b2d8-4f31-94fa-ee8bdf6967e7
126902025-04-30T06:44:13.169ZINFOtest_unauthorized (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc
126912025-04-30T06:44:13.235ZINFOtest_unauthorized (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc zone_address = ::1
126922025-04-30T06:44:13.235ZDEBGtest_unauthorized (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc uri = http://[::1]:12225/local/switch-id
126932025-04-30T06:44:13.235ZDEBGtest_unauthorized (MgsClient): client response background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
126942025-04-30T06:44:13.235ZWARNtest_unauthorized (BackgroundTasks): failed to identify switch slot for dendrite background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
126952025-04-30T06:44:13.240ZWARNtest_unauthorized (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc
126962025-04-30T06:44:13.297ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
126972025-04-30T06:44:13.298ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory
126982025-04-30T06:44:13.298ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
126992025-04-30T06:44:13.298ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
127002025-04-30T06:44:13.299ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
127012025-04-30T06:44:13.299ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
127022025-04-30T06:44:13.301ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
127032025-04-30T06:44:13.301ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:64132 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
127042025-04-30T06:44:13.301ZDEBGtest_unauthorized (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
127052025-04-30T06:44:13.301ZDEBGtest_unauthorized (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
127062025-04-30T06:44:13.301ZDEBGtest_unauthorized (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
127072025-04-30T06:44:13.301ZDEBGtest_unauthorized (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
127082025-04-30T06:44:13.302ZDEBGtest_unauthorized (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a
127092025-04-30T06:44:13.303ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
127102025-04-30T06:44:13.303ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:64132 req_id = cbc686f7-46d4-47aa-b02b-d0f13cdfed09 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
127112025-04-30T06:44:13.305ZINFOtest_unauthorized (dropshot): request completed latency_us = 1384 local_addr = [::1]:57786 method = GET remote_addr = [::1]:64132 req_id = cbc686f7-46d4-47aa-b02b-d0f13cdfed09 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
127122025-04-30T06:44:13.305ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
127132025-04-30T06:44:13.311ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
127142025-04-30T06:44:13.311ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
127152025-04-30T06:44:13.311ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
127162025-04-30T06:44:13.312ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
127172025-04-30T06:44:13.312ZTRACtest_unauthorized (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
127182025-04-30T06:44:13.315ZDEBGtest_unauthorized (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
127192025-04-30T06:44:13.315ZTRACtest_unauthorized (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
127202025-04-30T06:44:13.315ZTRACtest_unauthorized (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
127212025-04-30T06:44:13.315ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = http://[::1]:57786 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:57786/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "cbc686f7-46d4-47aa-b02b-d0f13cdfed09", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
127222025-04-30T06:44:13.321ZDEBGtest_unauthorized (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
127232025-04-30T06:44:13.321ZDEBGtest_unauthorized (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
127242025-04-30T06:44:13.321ZDEBGtest_unauthorized (BackgroundTasks): finished collection background_task = inventory_collection
127252025-04-30T06:44:13.327ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
127262025-04-30T06:44:13.327ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
127272025-04-30T06:44:13.327ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
127282025-04-30T06:44:13.327ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
127292025-04-30T06:44:13.328ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
127302025-04-30T06:44:13.338ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 2 table_name = fields_string
127312025-04-30T06:44:13.338ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
127322025-04-30T06:44:13.339ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
127332025-04-30T06:44:13.339ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
127342025-04-30T06:44:13.339ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
127352025-04-30T06:44:13.340ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
127362025-04-30T06:44:13.344ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
127372025-04-30T06:44:13.344ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
127382025-04-30T06:44:13.344ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
127392025-04-30T06:44:13.345ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
127402025-04-30T06:44:13.349ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 1 table_name = measurements_i64
127412025-04-30T06:44:13.349ZTRACtest_unauthorized (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
127422025-04-30T06:44:13.392ZWARNtest_unauthorized (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
127432025-04-30T06:44:13.392ZWARNtest_unauthorized (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
127442025-04-30T06:44:13.392ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 755.895883ms iteration = 45 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
127452025-04-30T06:44:13.392ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 46 reason = Signaled
127462025-04-30T06:44:13.392ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
127472025-04-30T06:44:13.392ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
127482025-04-30T06:44:13.393ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
127492025-04-30T06:44:13.393ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
127502025-04-30T06:44:13.394ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
127512025-04-30T06:44:13.407ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
127522025-04-30T06:44:13.413ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
127532025-04-30T06:44:13.413ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
127542025-04-30T06:44:13.413ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
127552025-04-30T06:44:13.414ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
127562025-04-30T06:44:13.415ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
127572025-04-30T06:44:13.415ZWARNtest_unauthorized (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
127582025-04-30T06:44:13.418ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1a8b2318-abf8-406d-a389-112ab91a0de8, 0)]
127592025-04-30T06:44:13.418ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
127602025-04-30T06:44:13.419ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 9b 71 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .q..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:59161 req_id = 1c5d9b90-7042-4e13-b04b-746d74631eb5
127612025-04-30T06:44:13.420ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 39793,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:59161 req_id = 1c5d9b90-7042-4e13-b04b-746d74631eb5
127622025-04-30T06:44:13.420ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
127632025-04-30T06:44:13.420ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
127642025-04-30T06:44:13.420ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:59161 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 1c5d9b90-7042-4e13-b04b-746d74631eb5
127652025-04-30T06:44:13.421ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4318, tv_nsec: 220170233 } })
127662025-04-30T06:44:13.421ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
127672025-04-30T06:44:13.421ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: a7 15 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:60042 req_id = 1871de20-5f11-4f4a-a761-5201801dcd83
127682025-04-30T06:44:13.421ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 42773,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60042 req_id = 1871de20-5f11-4f4a-a761-5201801dcd83
127692025-04-30T06:44:13.421ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
127702025-04-30T06:44:13.421ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:60042 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 1871de20-5f11-4f4a-a761-5201801dcd83
127712025-04-30T06:44:13.506ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
127722025-04-30T06:44:13.508ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
127732025-04-30T06:44:13.508ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
127742025-04-30T06:44:13.508ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
127752025-04-30T06:44:13.508ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
127762025-04-30T06:44:13.509ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
127772025-04-30T06:44:13.511ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 80 0f 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:57095 req_id = 59694301-c205-480d-9f73-bbe551547db3
127782025-04-30T06:44:13.511ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 32783,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:57095 req_id = 59694301-c205-480d-9f73-bbe551547db3
127792025-04-30T06:44:13.511ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
127802025-04-30T06:44:13.512ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 32783, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:57095 req_id = 59694301-c205-480d-9f73-bbe551547db3
127812025-04-30T06:44:13.512ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
127822025-04-30T06:44:13.512ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
127832025-04-30T06:44:13.512ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
127842025-04-30T06:44:13.515ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
127852025-04-30T06:44:13.523ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
127862025-04-30T06:44:13.523ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
127872025-04-30T06:44:13.523ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
127882025-04-30T06:44:13.524ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
127892025-04-30T06:44:13.525ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "224d27d9-d691-4c42-b3fc-0c3c370436eb", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
127902025-04-30T06:44:13.525ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
127912025-04-30T06:44:13.528ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b8a81de6-b0d7-48c7-8a2c-f847348bed49", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
127922025-04-30T06:44:13.528ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
127932025-04-30T06:44:13.528ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
127942025-04-30T06:44:13.528ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
127952025-04-30T06:44:13.529ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
127962025-04-30T06:44:13.531ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "42207e78-06c9-4c7a-a6ce-8cb7775cbb44", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
127972025-04-30T06:44:13.531ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
127982025-04-30T06:44:13.533ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2acc4a24-feca-4c3c-a287-76d146a078ef", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
127992025-04-30T06:44:13.534ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
128002025-04-30T06:44:13.543ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a9352ddc-e3ed-44e4-b360-ecea76d01e7a", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
128012025-04-30T06:44:13.543ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
128022025-04-30T06:44:13.571ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ced26b82-c6b5-4c13-aef6-e13c7c091d4b", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
128032025-04-30T06:44:13.571ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
128042025-04-30T06:44:13.574ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
128052025-04-30T06:44:13.576ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "97de4eb6-b92d-4176-9e76-f8b40fdc0a5d", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
128062025-04-30T06:44:13.576ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
128072025-04-30T06:44:13.577ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bbbe26b3-f5f6-433d-af4a-d83ddfe02227", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
128082025-04-30T06:44:13.577ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
128092025-04-30T06:44:13.578ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "56a33a26-e6c4-4db4-bcec-0e6f6614ebfe", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
128102025-04-30T06:44:13.579ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
128112025-04-30T06:44:13.579ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3cffd080-4cc1-4ae1-9fa4-4d65974f4a56", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
128122025-04-30T06:44:13.580ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
128132025-04-30T06:44:13.581ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a5996434-8b58-4ced-b8d0-f768964c9a07", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
128142025-04-30T06:44:13.581ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
128152025-04-30T06:44:13.582ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "236ed6b6-b06b-42f6-862e-51a56adc739d", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
128162025-04-30T06:44:13.582ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
128172025-04-30T06:44:13.584ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a91a61b8-d350-4d6f-91dc-55fad0087947", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
128182025-04-30T06:44:13.584ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
128192025-04-30T06:44:13.585ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "157ad18b-faeb-4279-8d01-68a282b3138b", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
128202025-04-30T06:44:13.585ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
128212025-04-30T06:44:13.588ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ba5e794a-f04b-49e6-b8e1-73d4dea0d682", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
128222025-04-30T06:44:13.588ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
128232025-04-30T06:44:13.589ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e64038a9-bee7-42a3-b6dd-8973a432cee2", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
128242025-04-30T06:44:13.589ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
128252025-04-30T06:44:13.592ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ba6d5683-8c86-4ebc-8a49-76b4cbe68a88", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
128262025-04-30T06:44:13.592ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
128272025-04-30T06:44:13.594ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "54aa8f1e-298d-47b0-b3c3-deb166a78eb2", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
128282025-04-30T06:44:13.595ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
128292025-04-30T06:44:13.597ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a10ec73c-7596-4d56-b1f1-30c0ac927cd8", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
128302025-04-30T06:44:13.597ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
128312025-04-30T06:44:13.598ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7b23e624-fd83-44e5-b480-e8dbfbf04f90", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
128322025-04-30T06:44:13.598ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
128332025-04-30T06:44:13.599ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5e51d109-be7a-4663-ab50-a00e5539a625", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
128342025-04-30T06:44:13.599ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
128352025-04-30T06:44:13.600ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c4eb462b-ea78-4e39-8293-93ce7509d440", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
128362025-04-30T06:44:13.600ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
128372025-04-30T06:44:13.601ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "500d1d2d-1f8f-4d01-9aa3-36a83d49e80a", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
128382025-04-30T06:44:13.602ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
128392025-04-30T06:44:13.603ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "be155e25-6c84-43ea-96b5-ec48299c6344", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
128402025-04-30T06:44:13.603ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
128412025-04-30T06:44:13.605ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1e1759dc-bbb0-4162-95cd-b8e694850456", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
128422025-04-30T06:44:13.605ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
128432025-04-30T06:44:13.607ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "30597da1-2b51-422e-86fa-82a8bd164c3b", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
128442025-04-30T06:44:13.608ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
128452025-04-30T06:44:13.611ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6c76b863-083b-4371-a959-519cb3179a53", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
128462025-04-30T06:44:13.611ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
128472025-04-30T06:44:13.613ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b5dd1c11-b053-405c-a790-3c64cb09aa4d", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
128482025-04-30T06:44:13.613ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
128492025-04-30T06:44:13.615ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ea290f2c-8bb7-4b05-bd2c-8b3e8298e8d1", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
128502025-04-30T06:44:13.615ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
128512025-04-30T06:44:13.619ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c90f2372-ca7b-4688-a922-28d07f9b325b", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
128522025-04-30T06:44:13.619ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
128532025-04-30T06:44:13.620ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4644c344-53eb-4f35-b62d-ce4a950fc906", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
128542025-04-30T06:44:13.620ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
128552025-04-30T06:44:13.622ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "85598195-86bb-4db1-ab49-749f787d1800", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
128562025-04-30T06:44:13.622ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
128572025-04-30T06:44:13.624ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1d02f005-c544-4829-824a-f0ce09b1cb68", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
128582025-04-30T06:44:13.624ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
128592025-04-30T06:44:13.625ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
128602025-04-30T06:44:13.625ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
128612025-04-30T06:44:13.625ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
128622025-04-30T06:44:13.625ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
128632025-04-30T06:44:13.627ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
128642025-04-30T06:44:13.628ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "495ecae5-ec6d-4549-b9c7-2c6b9996e309", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
128652025-04-30T06:44:13.628ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
128662025-04-30T06:44:13.630ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "33620e75-24c2-421c-baff-1679e3050065", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
128672025-04-30T06:44:13.631ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
128682025-04-30T06:44:13.631ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128692025-04-30T06:44:13.640ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
128702025-04-30T06:44:13.640ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
128712025-04-30T06:44:13.640ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
128722025-04-30T06:44:13.641ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
128732025-04-30T06:44:13.643ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a5f4f377-985f-4e54-b864-e3b53a55c079", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
128742025-04-30T06:44:13.643ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
128752025-04-30T06:44:13.644ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
128762025-04-30T06:44:13.644ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
128772025-04-30T06:44:13.644ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
128782025-04-30T06:44:13.647ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b86b6860-4712-40f8-aa96-041a7e71c3f1", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
128792025-04-30T06:44:13.647ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
128802025-04-30T06:44:13.651ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "010aa829-eb33-4605-9e22-65dcf6c20c0b", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
128812025-04-30T06:44:13.651ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
128822025-04-30T06:44:13.654ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2167cfe9-eda3-4f92-b77d-f77e82f15e57", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
128832025-04-30T06:44:13.654ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
128842025-04-30T06:44:13.656ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0af2d79f-9590-4a72-a65f-91de54c75061", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
128852025-04-30T06:44:13.656ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
128862025-04-30T06:44:13.663ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a94a813f-d8cf-41ea-bbba-a21bf793dc7f", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
128872025-04-30T06:44:13.663ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
128882025-04-30T06:44:13.674ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "26282a54-3238-4c6c-b6f0-ec6bb1171973", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
128892025-04-30T06:44:13.674ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
128902025-04-30T06:44:13.682ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8d37d5db-430a-460e-81f5-8eb5fedab600", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
128912025-04-30T06:44:13.682ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
128922025-04-30T06:44:13.684ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "390c1c86-278e-428a-b5ab-ef06cb257089", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
128932025-04-30T06:44:13.684ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
128942025-04-30T06:44:13.685ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cecbd247-fe90-4380-9498-e2d64254ad9e", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
128952025-04-30T06:44:13.686ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
128962025-04-30T06:44:13.686ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
128972025-04-30T06:44:13.686ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
128982025-04-30T06:44:13.686ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
128992025-04-30T06:44:13.687ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
129002025-04-30T06:44:13.692ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
129012025-04-30T06:44:13.693ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129022025-04-30T06:44:13.698ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
129032025-04-30T06:44:13.698ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
129042025-04-30T06:44:13.698ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
129052025-04-30T06:44:13.698ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
129062025-04-30T06:44:13.698ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
129072025-04-30T06:44:13.698ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
129082025-04-30T06:44:13.701ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129092025-04-30T06:44:13.706ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
129102025-04-30T06:44:13.706ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
129112025-04-30T06:44:13.706ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
129122025-04-30T06:44:13.708ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
129132025-04-30T06:44:13.789ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
129142025-04-30T06:44:13.790ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory
129152025-04-30T06:44:13.791ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
129162025-04-30T06:44:13.791ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
129172025-04-30T06:44:13.791ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
129182025-04-30T06:44:13.791ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
129192025-04-30T06:44:13.792ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
129202025-04-30T06:44:13.793ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:43656 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
129212025-04-30T06:44:13.794ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:43656 req_id = a50a17ba-7d37-4710-8c39-3c2c7f923017 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
129222025-04-30T06:44:13.795ZINFOtest_unauthorized (dropshot): request completed latency_us = 985 local_addr = [::1]:57786 method = GET remote_addr = [::1]:43656 req_id = a50a17ba-7d37-4710-8c39-3c2c7f923017 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
129232025-04-30T06:44:13.795ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = http://[::1]:57786 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:57786/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "a50a17ba-7d37-4710-8c39-3c2c7f923017", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
129242025-04-30T06:44:13.798ZDEBGtest_unauthorized (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
129252025-04-30T06:44:13.798ZDEBGtest_unauthorized (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
129262025-04-30T06:44:13.798ZDEBGtest_unauthorized (BackgroundTasks): finished collection background_task = inventory_collection
129272025-04-30T06:44:13.800ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
129282025-04-30T06:44:13.800ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
129292025-04-30T06:44:13.800ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
129302025-04-30T06:44:13.800ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
129312025-04-30T06:44:13.801ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
129322025-04-30T06:44:13.801ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129332025-04-30T06:44:13.815ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
129342025-04-30T06:44:13.815ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
129352025-04-30T06:44:13.815ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
129362025-04-30T06:44:13.816ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
129372025-04-30T06:44:13.819ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129382025-04-30T06:44:13.825ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
129392025-04-30T06:44:13.825ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
129402025-04-30T06:44:13.825ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
129412025-04-30T06:44:13.826ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
129422025-04-30T06:44:13.828ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
129432025-04-30T06:44:13.828ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
129442025-04-30T06:44:13.832ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
129452025-04-30T06:44:13.872ZWARNtest_unauthorized (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
129462025-04-30T06:44:13.874ZWARNtest_unauthorized (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
129472025-04-30T06:44:13.874ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 481.161132ms iteration = 46 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
129482025-04-30T06:44:13.874ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 47 reason = Signaled
129492025-04-30T06:44:13.874ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
129502025-04-30T06:44:13.874ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
129512025-04-30T06:44:13.874ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
129522025-04-30T06:44:13.874ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
129532025-04-30T06:44:13.875ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
129542025-04-30T06:44:13.879ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
129552025-04-30T06:44:13.880ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129562025-04-30T06:44:13.889ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
129572025-04-30T06:44:13.889ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
129582025-04-30T06:44:13.889ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
129592025-04-30T06:44:13.891ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
129602025-04-30T06:44:13.894ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1a8b2318-abf8-406d-a389-112ab91a0de8, 0)]
129612025-04-30T06:44:13.894ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
129622025-04-30T06:44:13.895ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 5b b0 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 [............_mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:55311 req_id = 0fb866c0-a9fa-437a-8620-e1e06c2cab7b
129632025-04-30T06:44:13.895ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 23472,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:55311 req_id = 0fb866c0-a9fa-437a-8620-e1e06c2cab7b
129642025-04-30T06:44:13.895ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
129652025-04-30T06:44:13.896ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
129662025-04-30T06:44:13.896ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:55311 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 0fb866c0-a9fa-437a-8620-e1e06c2cab7b
129672025-04-30T06:44:13.896ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4318, tv_nsec: 695609384 } })
129682025-04-30T06:44:13.896ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
129692025-04-30T06:44:13.897ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 80 1d 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:54842 req_id = fb7b9db8-0ec1-4362-a5f8-d2f76a066b4a
129702025-04-30T06:44:13.897ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 32797,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54842 req_id = fb7b9db8-0ec1-4362-a5f8-d2f76a066b4a
129712025-04-30T06:44:13.897ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
129722025-04-30T06:44:13.897ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:54842 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = fb7b9db8-0ec1-4362-a5f8-d2f76a066b4a
129732025-04-30T06:44:13.955ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
129742025-04-30T06:44:13.956ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
129752025-04-30T06:44:13.956ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
129762025-04-30T06:44:13.956ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
129772025-04-30T06:44:13.956ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
129782025-04-30T06:44:13.958ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
129792025-04-30T06:44:13.958ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 55 6d 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c Um..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:55667 req_id = d40f8f48-65a0-46c2-b883-bb606e4e718c
129802025-04-30T06:44:13.959ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 21869,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:55667 req_id = d40f8f48-65a0-46c2-b883-bb606e4e718c
129812025-04-30T06:44:13.959ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
129822025-04-30T06:44:13.959ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 21869, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:55667 req_id = d40f8f48-65a0-46c2-b883-bb606e4e718c
129832025-04-30T06:44:13.963ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
129842025-04-30T06:44:13.963ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
129852025-04-30T06:44:13.963ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
129862025-04-30T06:44:13.964ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129872025-04-30T06:44:13.971ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
129882025-04-30T06:44:13.971ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
129892025-04-30T06:44:13.971ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
129902025-04-30T06:44:13.972ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
129912025-04-30T06:44:13.979ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "a35d10c5-ec2b-43e3-8cfd-09c14e4579b3", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
129922025-04-30T06:44:13.981ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
129932025-04-30T06:44:13.982ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
129942025-04-30T06:44:13.982ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
129952025-04-30T06:44:13.984ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
129962025-04-30T06:44:13.997ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7cf5ecf6-0840-4594-a880-77b756afdd78", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
129972025-04-30T06:44:13.998ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
129982025-04-30T06:44:14.006ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9bfd8fd2-f5c1-478a-a3d3-806119d650bc", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
129992025-04-30T06:44:14.006ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
130002025-04-30T06:44:14.012ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5d7223e1-12f0-4f49-ac3c-7e1248a87b3c", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
130012025-04-30T06:44:14.012ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
130022025-04-30T06:44:14.017ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fab05e8c-ca70-491f-a8a1-5b54b45c45bf", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
130032025-04-30T06:44:14.017ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
130042025-04-30T06:44:14.023ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "44c9deff-a6a2-4c70-b970-018e76acd95d", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
130052025-04-30T06:44:14.023ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
130062025-04-30T06:44:14.027ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "470e74b2-cf37-4e48-a1ea-0d8f8b770ca7", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
130072025-04-30T06:44:14.027ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
130082025-04-30T06:44:14.031ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
130092025-04-30T06:44:14.033ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a29c0255-93c7-471a-abd0-143bd027edd7", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
130102025-04-30T06:44:14.033ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
130112025-04-30T06:44:14.034ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "62413a17-24db-4a35-808e-c25ad3e442d6", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
130122025-04-30T06:44:14.034ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
130132025-04-30T06:44:14.035ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "44677e55-8106-44bc-a648-4982e593022e", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
130142025-04-30T06:44:14.042ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
130152025-04-30T06:44:14.046ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "71c646b1-be39-4dc2-878a-424dcaa8493f", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
130162025-04-30T06:44:14.047ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
130172025-04-30T06:44:14.048ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "aabbc12c-6841-4fb8-8449-9c7ee5c63b90", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
130182025-04-30T06:44:14.048ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
130192025-04-30T06:44:14.051ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d39dfda5-0784-4196-a770-36c07f0cc1fa", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
130202025-04-30T06:44:14.051ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
130212025-04-30T06:44:14.053ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "55219f64-1f1e-4526-822d-216d43a22bbf", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
130222025-04-30T06:44:14.053ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
130232025-04-30T06:44:14.055ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ffdcafc3-23a1-41ac-a237-2ff926763119", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
130242025-04-30T06:44:14.055ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
130252025-04-30T06:44:14.058ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a8839330-b693-4835-9dd9-e3120761fdd3", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
130262025-04-30T06:44:14.058ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
130272025-04-30T06:44:14.060ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d9734519-1062-4a66-8069-479871c16ef4", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
130282025-04-30T06:44:14.060ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
130292025-04-30T06:44:14.062ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2d6d9fd5-323d-4833-8ace-328d4e6d1c64", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
130302025-04-30T06:44:14.062ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
130312025-04-30T06:44:14.065ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7a657069-edf6-4645-bd4a-608d1e03f066", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
130322025-04-30T06:44:14.065ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
130332025-04-30T06:44:14.067ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4e352021-1460-40fa-af3b-09b56130fb70", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
130342025-04-30T06:44:14.067ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
130352025-04-30T06:44:14.067ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1217749f-5269-49c4-bed2-a81a0daeeae8", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
130362025-04-30T06:44:14.067ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
130372025-04-30T06:44:14.068ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3e4c9a09-81cd-4ce8-8d21-2f3f7e0ebb4d", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
130382025-04-30T06:44:14.069ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
130392025-04-30T06:44:14.070ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4bc3f3ef-a2b3-41ef-9893-b66db98f3c10", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
130402025-04-30T06:44:14.070ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
130412025-04-30T06:44:14.072ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "073c9f20-5b2d-4103-b470-7a87f59c5fd2", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
130422025-04-30T06:44:14.072ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
130432025-04-30T06:44:14.075ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4f7070fe-2248-42c6-a7c6-4b284d81039a", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
130442025-04-30T06:44:14.075ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
130452025-04-30T06:44:14.079ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4fea4a38-b2f1-47ff-9281-2c41151193ad", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
130462025-04-30T06:44:14.079ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
130472025-04-30T06:44:14.082ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
130482025-04-30T06:44:14.082ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
130492025-04-30T06:44:14.082ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
130502025-04-30T06:44:14.082ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
130512025-04-30T06:44:14.088ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
130522025-04-30T06:44:14.090ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f44e7586-a34e-48a0-b611-1ea3a1d4dce7", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
130532025-04-30T06:44:14.091ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
130542025-04-30T06:44:14.091ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130552025-04-30T06:44:14.100ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
130562025-04-30T06:44:14.100ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
130572025-04-30T06:44:14.100ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
130582025-04-30T06:44:14.101ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
130592025-04-30T06:44:14.104ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
130602025-04-30T06:44:14.104ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
130612025-04-30T06:44:14.104ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
130622025-04-30T06:44:14.106ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2346cf9d-46e6-4842-8272-0f0f8e64b284", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
130632025-04-30T06:44:14.107ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
130642025-04-30T06:44:14.118ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "824bdfe8-81ff-4438-ad1a-0eda399e17b7", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
130652025-04-30T06:44:14.119ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
130662025-04-30T06:44:14.120ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "04949d25-2058-4e92-90fc-695aa7aecb27", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
130672025-04-30T06:44:14.120ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
130682025-04-30T06:44:14.122ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8e4fa83b-8a1f-4aac-afb6-ff3a4bb9a3c2", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
130692025-04-30T06:44:14.123ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
130702025-04-30T06:44:14.127ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7a76daa4-0abe-40fa-bf9e-9d0674d6126d", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
130712025-04-30T06:44:14.128ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
130722025-04-30T06:44:14.129ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3dec3042-8661-4bea-bcd4-83351b375af4", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
130732025-04-30T06:44:14.129ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
130742025-04-30T06:44:14.130ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "09f4cbe9-36a1-4725-80ae-3313e2b26e83", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
130752025-04-30T06:44:14.130ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
130762025-04-30T06:44:14.132ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "838c3520-daec-44aa-ad42-25c31f240090", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
130772025-04-30T06:44:14.132ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
130782025-04-30T06:44:14.134ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0be260d0-4a37-4fed-b11e-67997e887457", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
130792025-04-30T06:44:14.134ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
130802025-04-30T06:44:14.144ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "af602e23-3855-4f6a-bf0f-af331b6e3b78", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
130812025-04-30T06:44:14.145ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
130822025-04-30T06:44:14.150ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "60dfe5ff-8591-40eb-ad39-6c0713203739", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
130832025-04-30T06:44:14.150ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
130842025-04-30T06:44:14.151ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
130852025-04-30T06:44:14.153ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "294c9c4d-693a-4062-87e3-3305ae0f61f1", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
130862025-04-30T06:44:14.153ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
130872025-04-30T06:44:14.155ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a746ea4c-8659-413a-8bae-55fdb0e60b61", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
130882025-04-30T06:44:14.155ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
130892025-04-30T06:44:14.158ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fe25cf54-6c75-4783-a141-9f511c9cc597", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
130902025-04-30T06:44:14.158ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
130912025-04-30T06:44:14.159ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "78940be9-489f-41af-abbe-62b9e552c9d1", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
130922025-04-30T06:44:14.159ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
130932025-04-30T06:44:14.161ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4224b63c-0955-4a49-b4ee-8723335d161a", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
130942025-04-30T06:44:14.161ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
130952025-04-30T06:44:14.164ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c4abb16b-613d-41a1-a2e3-db09fd367fe1", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
130962025-04-30T06:44:14.164ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
130972025-04-30T06:44:14.165ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5033d5e0-6e67-47cc-b116-3561e1f5f9f9", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
130982025-04-30T06:44:14.165ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
130992025-04-30T06:44:14.165ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
131002025-04-30T06:44:14.165ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
131012025-04-30T06:44:14.165ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
131022025-04-30T06:44:14.166ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
131032025-04-30T06:44:14.169ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131042025-04-30T06:44:14.174ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
131052025-04-30T06:44:14.174ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
131062025-04-30T06:44:14.174ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
131072025-04-30T06:44:14.175ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
131082025-04-30T06:44:14.175ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
131092025-04-30T06:44:14.176ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
131102025-04-30T06:44:14.176ZDEBGtest_unauthorized (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
131112025-04-30T06:44:14.177ZDEBGtest_unauthorized (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
131122025-04-30T06:44:14.177ZDEBGtest_unauthorized (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a
131132025-04-30T06:44:14.179ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
131142025-04-30T06:44:14.183ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131152025-04-30T06:44:14.190ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
131162025-04-30T06:44:14.190ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
131172025-04-30T06:44:14.191ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
131182025-04-30T06:44:14.191ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
131192025-04-30T06:44:14.193ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 2 table_name = fields_string
131202025-04-30T06:44:14.193ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
131212025-04-30T06:44:14.277ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
131222025-04-30T06:44:14.277ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory
131232025-04-30T06:44:14.278ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
131242025-04-30T06:44:14.278ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
131252025-04-30T06:44:14.278ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
131262025-04-30T06:44:14.278ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
131272025-04-30T06:44:14.280ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
131282025-04-30T06:44:14.280ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:62146 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
131292025-04-30T06:44:14.281ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 1 table_name = measurements_i64
131302025-04-30T06:44:14.282ZTRACtest_unauthorized (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
131312025-04-30T06:44:14.283ZDEBGtest_unauthorized (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
131322025-04-30T06:44:14.283ZDEBGtest_unauthorized (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
131332025-04-30T06:44:14.283ZWARNtest_unauthorized (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
131342025-04-30T06:44:14.284ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:62146 req_id = cf56eef8-879b-4eb9-b467-25ac1820b409 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
131352025-04-30T06:44:14.286ZINFOtest_unauthorized (dropshot): request completed latency_us = 1350 local_addr = [::1]:57786 method = GET remote_addr = [::1]:62146 req_id = cf56eef8-879b-4eb9-b467-25ac1820b409 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
131362025-04-30T06:44:14.286ZTRACtest_unauthorized (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
131372025-04-30T06:44:14.287ZDEBGtest_unauthorized (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
131382025-04-30T06:44:14.287ZTRACtest_unauthorized (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
131392025-04-30T06:44:14.287ZTRACtest_unauthorized (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
131402025-04-30T06:44:14.287ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = http://[::1]:57786 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:57786/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "cf56eef8-879b-4eb9-b467-25ac1820b409", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
131412025-04-30T06:44:14.291ZDEBGtest_unauthorized (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
131422025-04-30T06:44:14.291ZDEBGtest_unauthorized (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
131432025-04-30T06:44:14.291ZDEBGtest_unauthorized (BackgroundTasks): finished collection background_task = inventory_collection
131442025-04-30T06:44:14.295ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
131452025-04-30T06:44:14.295ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
131462025-04-30T06:44:14.295ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
131472025-04-30T06:44:14.295ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
131482025-04-30T06:44:14.296ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
131492025-04-30T06:44:14.296ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131502025-04-30T06:44:14.304ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
131512025-04-30T06:44:14.304ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
131522025-04-30T06:44:14.304ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
131532025-04-30T06:44:14.305ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
131542025-04-30T06:44:14.308ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131552025-04-30T06:44:14.313ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
131562025-04-30T06:44:14.313ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
131572025-04-30T06:44:14.313ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
131582025-04-30T06:44:14.314ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
131592025-04-30T06:44:14.315ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
131602025-04-30T06:44:14.315ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
131612025-04-30T06:44:14.315ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
131622025-04-30T06:44:14.342ZWARNtest_unauthorized (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
131632025-04-30T06:44:14.343ZWARNtest_unauthorized (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
131642025-04-30T06:44:14.343ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 468.900453ms iteration = 47 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
131652025-04-30T06:44:14.343ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 48 reason = Signaled
131662025-04-30T06:44:14.343ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
131672025-04-30T06:44:14.343ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
131682025-04-30T06:44:14.343ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
131692025-04-30T06:44:14.343ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
131702025-04-30T06:44:14.344ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
131712025-04-30T06:44:14.347ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131722025-04-30T06:44:14.353ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
131732025-04-30T06:44:14.353ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
131742025-04-30T06:44:14.353ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
131752025-04-30T06:44:14.354ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
131762025-04-30T06:44:14.358ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1a8b2318-abf8-406d-a389-112ab91a0de8, 0)]
131772025-04-30T06:44:14.358ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
131782025-04-30T06:44:14.359ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
131792025-04-30T06:44:14.360ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 32 c3 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 2............_mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:49190 req_id = 532101f8-8644-4dd0-a04c-bae1e1713b8c
131802025-04-30T06:44:14.360ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 12995,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:49190 req_id = 532101f8-8644-4dd0-a04c-bae1e1713b8c
131812025-04-30T06:44:14.360ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
131822025-04-30T06:44:14.361ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
131832025-04-30T06:44:14.361ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:49190 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 532101f8-8644-4dd0-a04c-bae1e1713b8c
131842025-04-30T06:44:14.362ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4319, tv_nsec: 160998211 } })
131852025-04-30T06:44:14.362ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
131862025-04-30T06:44:14.362ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 9c bf 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:54923 req_id = e0809423-012a-4b1c-b250-b5d80bed4e32
131872025-04-30T06:44:14.362ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 40127,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54923 req_id = e0809423-012a-4b1c-b250-b5d80bed4e32
131882025-04-30T06:44:14.363ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
131892025-04-30T06:44:14.363ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:54923 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = e0809423-012a-4b1c-b250-b5d80bed4e32
131902025-04-30T06:44:14.424ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
131912025-04-30T06:44:14.425ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
131922025-04-30T06:44:14.425ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
131932025-04-30T06:44:14.425ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
131942025-04-30T06:44:14.425ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
131952025-04-30T06:44:14.425ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
131962025-04-30T06:44:14.427ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: dc f6 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:61643 req_id = caffdc5a-7d3a-4cf2-b8a0-dce0e7945d7b
131972025-04-30T06:44:14.427ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 56566,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:61643 req_id = caffdc5a-7d3a-4cf2-b8a0-dce0e7945d7b
131982025-04-30T06:44:14.429ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
131992025-04-30T06:44:14.429ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 56566, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:61643 req_id = caffdc5a-7d3a-4cf2-b8a0-dce0e7945d7b
132002025-04-30T06:44:14.429ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
132012025-04-30T06:44:14.429ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
132022025-04-30T06:44:14.429ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
132032025-04-30T06:44:14.430ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132042025-04-30T06:44:14.454ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
132052025-04-30T06:44:14.454ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
132062025-04-30T06:44:14.454ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
132072025-04-30T06:44:14.455ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
132082025-04-30T06:44:14.459ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "df37f855-1eb6-42b9-8dbb-1f3efeffed26", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
132092025-04-30T06:44:14.460ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
132102025-04-30T06:44:14.462ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
132112025-04-30T06:44:14.462ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
132122025-04-30T06:44:14.464ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
132132025-04-30T06:44:14.466ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4671731b-a821-417b-86e3-c735d4d5e97c", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
132142025-04-30T06:44:14.466ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
132152025-04-30T06:44:14.475ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1e9d1e6f-aeeb-4a31-af48-052859ba0762", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
132162025-04-30T06:44:14.475ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
132172025-04-30T06:44:14.482ZINFOtest_unauthorized (dropshot): accepted connection kind = pantry local_addr = [::1]:41451 remote_addr = [::1]:54898 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
132182025-04-30T06:44:14.483ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "02f667c2-2260-40ef-ae0b-25b93bf2f5c8", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
132192025-04-30T06:44:14.483ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
132202025-04-30T06:44:14.484ZTRACtest_unauthorized (dropshot): incoming request kind = pantry local_addr = [::1]:41451 method = GET remote_addr = [::1]:54898 req_id = a40aa305-ee3d-415a-b933-0e6956f8532a sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
132212025-04-30T06:44:14.484ZINFOtest_unauthorized (dropshot): request completed kind = pantry latency_us = 421 local_addr = [::1]:41451 method = GET remote_addr = [::1]:54898 req_id = a40aa305-ee3d-415a-b933-0e6956f8532a response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
132222025-04-30T06:44:14.486ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ba424b7c-075d-4ef5-a04e-6406aee09ee8", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
132232025-04-30T06:44:14.486ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
132242025-04-30T06:44:14.497ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "820abd7f-0e30-469e-987b-a48765b18a53", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
132252025-04-30T06:44:14.498ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
132262025-04-30T06:44:14.502ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d94e8ffc-1c82-43c2-82a9-aa056b21f658", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
132272025-04-30T06:44:14.503ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
132282025-04-30T06:44:14.506ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
132292025-04-30T06:44:14.514ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a3e4598-9ea7-4406-92c9-f55af1920df2", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
132302025-04-30T06:44:14.514ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
132312025-04-30T06:44:14.516ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "55a175fc-daa9-424a-bea1-891f61143bee", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
132322025-04-30T06:44:14.517ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
132332025-04-30T06:44:14.523ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ebe9b470-4d9c-4268-91a3-0eaecf3fa0d9", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
132342025-04-30T06:44:14.523ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
132352025-04-30T06:44:14.526ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9ff6cd58-e78e-4ed2-add7-c0852eb184b3", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
132362025-04-30T06:44:14.526ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
132372025-04-30T06:44:14.528ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "43bdd1fa-fc75-468d-a541-9f4f860929e0", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
132382025-04-30T06:44:14.528ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
132392025-04-30T06:44:14.532ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e7e96012-533a-4aaf-9b4a-6c5b69fd865a", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
132402025-04-30T06:44:14.532ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
132412025-04-30T06:44:14.557ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
132422025-04-30T06:44:14.557ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
132432025-04-30T06:44:14.557ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
132442025-04-30T06:44:14.557ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
132452025-04-30T06:44:14.558ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
132462025-04-30T06:44:14.589ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132472025-04-30T06:44:14.597ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
132482025-04-30T06:44:14.597ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
132492025-04-30T06:44:14.597ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
132502025-04-30T06:44:14.598ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
132512025-04-30T06:44:14.601ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1d4fc521-1aa3-484e-a1d6-52f1e2ca1b7f", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
132522025-04-30T06:44:14.601ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
132532025-04-30T06:44:14.606ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
132542025-04-30T06:44:14.606ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
132552025-04-30T06:44:14.607ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
132562025-04-30T06:44:14.613ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2c5f6687-0aa5-4c81-a0f3-44029eb136ce", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
132572025-04-30T06:44:14.613ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
132582025-04-30T06:44:14.615ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ff73e6d5-e5f7-4244-999f-71e32aa12538", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
132592025-04-30T06:44:14.615ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
132602025-04-30T06:44:14.617ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bde0765d-affa-4126-be78-6e0e2515c195", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
132612025-04-30T06:44:14.617ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
132622025-04-30T06:44:14.620ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f54cf37c-649d-4106-85cb-c0a0cc3c0db8", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
132632025-04-30T06:44:14.620ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
132642025-04-30T06:44:14.623ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "de5755b9-70e1-4247-bf66-38a85cdf03f6", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
132652025-04-30T06:44:14.623ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
132662025-04-30T06:44:14.623ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4ce142be-e6a4-457b-8fd4-fa33be934d0e", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
132672025-04-30T06:44:14.623ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
132682025-04-30T06:44:14.624ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "32fa5825-398b-48cd-a587-45a2b2f95dfe", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
132692025-04-30T06:44:14.624ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
132702025-04-30T06:44:14.625ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b1a671f6-ea51-458e-a842-1128005920af", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
132712025-04-30T06:44:14.625ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
132722025-04-30T06:44:14.626ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4b0f927c-4af6-4ad5-85c9-9b1cf830af21", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
132732025-04-30T06:44:14.627ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
132742025-04-30T06:44:14.628ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "082cd10e-5952-4e98-a292-4872c9d4187c", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
132752025-04-30T06:44:14.628ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
132762025-04-30T06:44:14.631ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ae9a17b4-bf85-47d0-aa13-7e6a31a92f38", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
132772025-04-30T06:44:14.631ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
132782025-04-30T06:44:14.634ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c1ab26ef-a875-481b-9e08-16bc4f12781b", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
132792025-04-30T06:44:14.634ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
132802025-04-30T06:44:14.636ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "236c102a-dc7d-4e44-928c-7b39075ca4aa", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
132812025-04-30T06:44:14.636ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
132822025-04-30T06:44:14.642ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
132832025-04-30T06:44:14.644ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f61d6ec1-a8e3-45f0-bc69-89c962fd26ae", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
132842025-04-30T06:44:14.644ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
132852025-04-30T06:44:14.647ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d138056e-d5a4-447c-897f-9995d98acb39", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
132862025-04-30T06:44:14.650ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
132872025-04-30T06:44:14.650ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d6588cd8-182d-4924-8865-e3cd1e6bfac5", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
132882025-04-30T06:44:14.651ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
132892025-04-30T06:44:14.652ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "73088bac-40fb-4440-8556-3b500c8fad12", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
132902025-04-30T06:44:14.652ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
132912025-04-30T06:44:14.655ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "063b0028-67eb-4b7c-845f-f14a06dbb313", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
132922025-04-30T06:44:14.655ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
132932025-04-30T06:44:14.656ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4950bb32-1449-4142-8c69-001da70ade5e", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
132942025-04-30T06:44:14.657ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
132952025-04-30T06:44:14.660ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7f62432f-72a2-49cf-8759-0403ed132346", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
132962025-04-30T06:44:14.660ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
132972025-04-30T06:44:14.661ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6dd3babb-3e94-4861-8a31-47dbfab236fe", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
132982025-04-30T06:44:14.661ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
132992025-04-30T06:44:14.665ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "560ca5c8-6733-4dae-b07c-cea476a45286", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
133002025-04-30T06:44:14.665ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
133012025-04-30T06:44:14.667ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6c9c788c-0215-46cc-94da-a11ef31452fe", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
133022025-04-30T06:44:14.667ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
133032025-04-30T06:44:14.669ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4f276eba-b2f9-4778-a081-f58e9b0c037d", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
133042025-04-30T06:44:14.669ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
133052025-04-30T06:44:14.673ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "639546a1-0f66-47a7-9dd4-a7bdbb7b8abe", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
133062025-04-30T06:44:14.674ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
133072025-04-30T06:44:14.680ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "baabae8b-dbe3-4b1f-b25d-0dbd74f2bb7c", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
133082025-04-30T06:44:14.680ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
133092025-04-30T06:44:14.684ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c11b241b-99b7-4f03-8863-11d6fc9427ea", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
133102025-04-30T06:44:14.685ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
133112025-04-30T06:44:14.686ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b7261543-c609-4f70-8b34-871994fed3fe", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
133122025-04-30T06:44:14.686ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
133132025-04-30T06:44:14.689ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd79752e-2912-44f4-8c9c-2fed3082ed5d", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
133142025-04-30T06:44:14.689ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
133152025-04-30T06:44:14.690ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d3ae6436-9002-45bf-870c-94ada18e0ccd", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
133162025-04-30T06:44:14.691ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
133172025-04-30T06:44:14.692ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9057f3da-25fe-45ed-8242-10851a1e6f54", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
133182025-04-30T06:44:14.692ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
133192025-04-30T06:44:14.692ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
133202025-04-30T06:44:14.692ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
133212025-04-30T06:44:14.692ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
133222025-04-30T06:44:14.693ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
133232025-04-30T06:44:14.694ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
133242025-04-30T06:44:14.694ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
133252025-04-30T06:44:14.694ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
133262025-04-30T06:44:14.694ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
133272025-04-30T06:44:14.695ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
133282025-04-30T06:44:14.697ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133292025-04-30T06:44:14.702ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
133302025-04-30T06:44:14.702ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
133312025-04-30T06:44:14.702ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
133322025-04-30T06:44:14.702ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
133332025-04-30T06:44:14.702ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
133342025-04-30T06:44:14.703ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
133352025-04-30T06:44:14.703ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133362025-04-30T06:44:14.708ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
133372025-04-30T06:44:14.708ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
133382025-04-30T06:44:14.708ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
133392025-04-30T06:44:14.709ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
133402025-04-30T06:44:14.710ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133412025-04-30T06:44:14.714ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
133422025-04-30T06:44:14.714ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
133432025-04-30T06:44:14.714ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
133442025-04-30T06:44:14.714ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
133452025-04-30T06:44:14.714ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
133462025-04-30T06:44:14.715ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
133472025-04-30T06:44:14.716ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
133482025-04-30T06:44:14.796ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
133492025-04-30T06:44:14.796ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory
133502025-04-30T06:44:14.796ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:38565 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
133512025-04-30T06:44:14.797ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:38565 req_id = 9d4cd1e2-48c7-4a7f-b458-3252a7822c2e sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
133522025-04-30T06:44:14.798ZINFOtest_unauthorized (dropshot): request completed latency_us = 485 local_addr = [::1]:57786 method = GET remote_addr = [::1]:38565 req_id = 9d4cd1e2-48c7-4a7f-b458-3252a7822c2e response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
133532025-04-30T06:44:14.799ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = http://[::1]:57786 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:57786/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "9d4cd1e2-48c7-4a7f-b458-3252a7822c2e", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
133542025-04-30T06:44:14.801ZDEBGtest_unauthorized (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
133552025-04-30T06:44:14.801ZDEBGtest_unauthorized (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
133562025-04-30T06:44:14.802ZDEBGtest_unauthorized (BackgroundTasks): finished collection background_task = inventory_collection
133572025-04-30T06:44:14.805ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
133582025-04-30T06:44:14.806ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
133592025-04-30T06:44:14.806ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
133602025-04-30T06:44:14.806ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
133612025-04-30T06:44:14.807ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
133622025-04-30T06:44:14.810ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133632025-04-30T06:44:14.819ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
133642025-04-30T06:44:14.819ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
133652025-04-30T06:44:14.820ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
133662025-04-30T06:44:14.821ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
133672025-04-30T06:44:14.847ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
133682025-04-30T06:44:14.869ZWARNtest_unauthorized (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
133692025-04-30T06:44:14.870ZWARNtest_unauthorized (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
133702025-04-30T06:44:14.870ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 527.558947ms iteration = 48 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
133712025-04-30T06:44:14.870ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 49 reason = Signaled
133722025-04-30T06:44:14.870ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
133732025-04-30T06:44:14.870ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
133742025-04-30T06:44:14.870ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
133752025-04-30T06:44:14.870ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
133762025-04-30T06:44:14.872ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
133772025-04-30T06:44:14.875ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133782025-04-30T06:44:14.881ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
133792025-04-30T06:44:14.881ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
133802025-04-30T06:44:14.881ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
133812025-04-30T06:44:14.881ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
133822025-04-30T06:44:14.885ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1a8b2318-abf8-406d-a389-112ab91a0de8, 0)]
133832025-04-30T06:44:14.885ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
133842025-04-30T06:44:14.887ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 18 8a 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:61932 req_id = 70d94fc0-3ed2-4c63-8ffe-2afac17a1040
133852025-04-30T06:44:14.887ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 6282,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:61932 req_id = 70d94fc0-3ed2-4c63-8ffe-2afac17a1040
133862025-04-30T06:44:14.887ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
133872025-04-30T06:44:14.887ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
133882025-04-30T06:44:14.887ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:61932 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 70d94fc0-3ed2-4c63-8ffe-2afac17a1040
133892025-04-30T06:44:14.888ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4319, tv_nsec: 687469992 } })
133902025-04-30T06:44:14.888ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
133912025-04-30T06:44:14.889ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 09 df 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:52228 req_id = 91baa0e8-2a57-4bf9-a9d6-78fa69572e2f
133922025-04-30T06:44:14.889ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 2527,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52228 req_id = 91baa0e8-2a57-4bf9-a9d6-78fa69572e2f
133932025-04-30T06:44:14.889ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
133942025-04-30T06:44:14.889ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:52228 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 91baa0e8-2a57-4bf9-a9d6-78fa69572e2f
133952025-04-30T06:44:14.955ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
133962025-04-30T06:44:14.956ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
133972025-04-30T06:44:14.956ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
133982025-04-30T06:44:14.956ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
133992025-04-30T06:44:14.956ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
134002025-04-30T06:44:14.957ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
134012025-04-30T06:44:14.958ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 7e b6 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ~............_cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:53297 req_id = ff2d89a5-6ff3-4ed6-ae62-35e56f0e1459
134022025-04-30T06:44:14.959ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 32438,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:53297 req_id = ff2d89a5-6ff3-4ed6-ae62-35e56f0e1459
134032025-04-30T06:44:14.960ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
134042025-04-30T06:44:14.960ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 32438, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:53297 req_id = ff2d89a5-6ff3-4ed6-ae62-35e56f0e1459
134052025-04-30T06:44:14.960ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
134062025-04-30T06:44:14.960ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
134072025-04-30T06:44:14.960ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
134082025-04-30T06:44:14.961ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134092025-04-30T06:44:14.968ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
134102025-04-30T06:44:14.968ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
134112025-04-30T06:44:14.968ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
134122025-04-30T06:44:14.969ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
134132025-04-30T06:44:14.970ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "7ef21bf9-263c-4cc6-b6f2-0203926aabaf", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
134142025-04-30T06:44:14.972ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
134152025-04-30T06:44:14.974ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
134162025-04-30T06:44:14.974ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
134172025-04-30T06:44:14.975ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
134182025-04-30T06:44:14.984ZTRACtest_unauthorized (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
134192025-04-30T06:44:14.984ZTRACtest_unauthorized (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 72 19 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 r............_de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:63758 req_id = 695c1d2f-88dd-41b6-9f4d-31908b701903
134202025-04-30T06:44:14.984ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 29209,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63758 req_id = 695c1d2f-88dd-41b6-9f4d-31908b701903
134212025-04-30T06:44:14.984ZDEBGtest_unauthorized (store): query key key = _dendrite._tcp
134222025-04-30T06:44:14.984ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
134232025-04-30T06:44:14.984ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:63758 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 44264, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 695c1d2f-88dd-41b6-9f4d-31908b701903
134242025-04-30T06:44:14.984ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3e299b69-fe58-4148-b539-cd3c2de17750", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
134252025-04-30T06:44:14.985ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
134262025-04-30T06:44:14.985ZDEBGtest_unauthorized (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 44264, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4319, tv_nsec: 779302352 } })
134272025-04-30T06:44:14.985ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
134282025-04-30T06:44:14.985ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: cd 66 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .f..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:52338 req_id = 99e9dd5e-599a-43a0-b39e-f26b9ff84d2f
134292025-04-30T06:44:14.985ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 52582,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52338 req_id = 99e9dd5e-599a-43a0-b39e-f26b9ff84d2f
134302025-04-30T06:44:14.985ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
134312025-04-30T06:44:14.985ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:52338 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 99e9dd5e-599a-43a0-b39e-f26b9ff84d2f
134322025-04-30T06:44:14.985ZINFOtest_unauthorized (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector
134332025-04-30T06:44:15.060ZINFOtest_unauthorized (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector zone_address = ::1
134342025-04-30T06:44:15.060ZDEBGtest_unauthorized (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[::1]:12225/local/switch-id
134352025-04-30T06:44:15.061ZDEBGtest_unauthorized (MgsClient): client response background_task = nat_v4_garbage_collector result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
134362025-04-30T06:44:15.061ZWARNtest_unauthorized (BackgroundTasks): failed to identify switch slot for dendrite background_task = nat_v4_garbage_collector reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
134372025-04-30T06:44:15.064ZWARNtest_unauthorized (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = nat_v4_garbage_collector
134382025-04-30T06:44:15.624ZINFOtest_unauthorized (dropshot): accepted connection kind = pantry local_addr = [::1]:41451 remote_addr = [::1]:37024 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
134392025-04-30T06:44:15.624ZINFOtest_unauthorized (dropshot): accepted connection kind = pantry local_addr = [::1]:41451 remote_addr = [::1]:39188 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
134402025-04-30T06:44:15.624ZINFOtest_unauthorized (dropshot): accepted connection kind = pantry local_addr = [::1]:41451 remote_addr = [::1]:41693 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
134412025-04-30T06:44:15.624ZINFOtest_unauthorized (dropshot): accepted connection kind = pantry local_addr = [::1]:41451 remote_addr = [::1]:39461 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
134422025-04-30T06:44:15.624ZINFOtest_unauthorized (dropshot): accepted connection kind = pantry local_addr = [::1]:41451 remote_addr = [::1]:61826 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
134432025-04-30T06:44:15.624ZINFOtest_unauthorized (dropshot): accepted connection kind = pantry local_addr = [::1]:41451 remote_addr = [::1]:56138 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
134442025-04-30T06:44:15.624ZINFOtest_unauthorized (dropshot): accepted connection kind = pantry local_addr = [::1]:41451 remote_addr = [::1]:49375 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
134452025-04-30T06:44:15.625ZTRACtest_unauthorized (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
134462025-04-30T06:44:15.625ZINFOtest_unauthorized (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
134472025-04-30T06:44:15.625ZDEBGtest_unauthorized (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
134482025-04-30T06:44:15.625ZDEBGtest_unauthorized (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
134492025-04-30T06:44:15.625ZDEBGtest_unauthorized (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
134502025-04-30T06:44:15.625ZTRACtest_unauthorized (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
134512025-04-30T06:44:15.626ZDEBGtest_unauthorized (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
134522025-04-30T06:44:15.627ZTRACtest_unauthorized (dropshot): incoming request kind = pantry local_addr = [::1]:41451 method = GET remote_addr = [::1]:37024 req_id = 6595a225-a2af-4d9b-ae25-d86b05baf776 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
134532025-04-30T06:44:15.627ZTRACtest_unauthorized (dropshot): incoming request kind = pantry local_addr = [::1]:41451 method = GET remote_addr = [::1]:39188 req_id = b182f15f-58d0-402b-8351-9c7192cd6ab8 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
134542025-04-30T06:44:15.627ZTRACtest_unauthorized (dropshot): incoming request kind = pantry local_addr = [::1]:41451 method = GET remote_addr = [::1]:41693 req_id = b131ae81-3497-4272-aa9f-50c1a758da43 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
134552025-04-30T06:44:15.627ZTRACtest_unauthorized (dropshot): incoming request kind = pantry local_addr = [::1]:41451 method = GET remote_addr = [::1]:39461 req_id = b527f365-4b61-4dae-b27f-2a00d8a84533 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
134562025-04-30T06:44:15.628ZDEBGtest_unauthorized (oximeter): client request body = None method = GET uri = http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
134572025-04-30T06:44:15.628ZDEBGtest_unauthorized (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a
134582025-04-30T06:44:15.629ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
134592025-04-30T06:44:15.631ZINFOtest_unauthorized (dropshot): request completed kind = pantry latency_us = 3847 local_addr = [::1]:41451 method = GET remote_addr = [::1]:37024 req_id = 6595a225-a2af-4d9b-ae25-d86b05baf776 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
134602025-04-30T06:44:15.631ZINFOtest_unauthorized (dropshot): request completed kind = pantry latency_us = 3856 local_addr = [::1]:41451 method = GET remote_addr = [::1]:39188 req_id = b182f15f-58d0-402b-8351-9c7192cd6ab8 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
134612025-04-30T06:44:15.631ZINFOtest_unauthorized (dropshot): request completed kind = pantry latency_us = 3872 local_addr = [::1]:41451 method = GET remote_addr = [::1]:41693 req_id = b131ae81-3497-4272-aa9f-50c1a758da43 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
134622025-04-30T06:44:15.631ZINFOtest_unauthorized (dropshot): request completed kind = pantry latency_us = 3839 local_addr = [::1]:41451 method = GET remote_addr = [::1]:39461 req_id = b527f365-4b61-4dae-b27f-2a00d8a84533 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
134632025-04-30T06:44:15.632ZTRACtest_unauthorized (dropshot): incoming request kind = pantry local_addr = [::1]:41451 method = GET remote_addr = [::1]:61826 req_id = f3d86888-6fbb-40a5-8cfb-ff4bf41a6c43 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
134642025-04-30T06:44:15.632ZTRACtest_unauthorized (dropshot): incoming request kind = pantry local_addr = [::1]:41451 method = GET remote_addr = [::1]:56138 req_id = 727e1c51-10e0-4c6e-bc5a-676bf3f5ae95 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
134652025-04-30T06:44:15.632ZTRACtest_unauthorized (dropshot): incoming request kind = pantry local_addr = [::1]:41451 method = GET remote_addr = [::1]:49375 req_id = 0a944a4a-9e3d-42b0-96da-f0d7005e54d5 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
134662025-04-30T06:44:15.633ZTRACtest_unauthorized (dropshot_internal): incoming request local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 119604c6-3cbf-4995-83b7-bb670ee98bca uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
134672025-04-30T06:44:15.634ZTRACtest_unauthorized (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 04 e9 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 ............._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:53329 req_id = bedbe115-355d-4ac3-80c9-414acf2eccfd
134682025-04-30T06:44:15.635ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 1257,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:53329 req_id = bedbe115-355d-4ac3-80c9-414acf2eccfd
134692025-04-30T06:44:15.635ZDEBGtest_unauthorized (store): query key key = _dendrite._tcp
134702025-04-30T06:44:15.636ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
134712025-04-30T06:44:15.636ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:53329 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 44264, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = bedbe115-355d-4ac3-80c9-414acf2eccfd
134722025-04-30T06:44:15.636ZTRACtest_unauthorized (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 43 7a 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 Cz..........._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:61185 req_id = d417c773-43ce-42f8-b112-a3f038d41af3
134732025-04-30T06:44:15.636ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 17274,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:61185 req_id = d417c773-43ce-42f8-b112-a3f038d41af3
134742025-04-30T06:44:15.636ZDEBGtest_unauthorized (store): query key key = _dendrite._tcp
134752025-04-30T06:44:15.637ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
134762025-04-30T06:44:15.637ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:61185 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 44264, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = d417c773-43ce-42f8-b112-a3f038d41af3
134772025-04-30T06:44:15.638ZTRACtest_unauthorized (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 119604c6-3cbf-4995-83b7-bb670ee98bca resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
134782025-04-30T06:44:15.638ZDEBGtest_unauthorized (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 119604c6-3cbf-4995-83b7-bb670ee98bca roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
134792025-04-30T06:44:15.639ZDEBGtest_unauthorized (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 119604c6-3cbf-4995-83b7-bb670ee98bca resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
134802025-04-30T06:44:15.639ZTRACtest_unauthorized (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
134812025-04-30T06:44:15.640ZDEBGtest_unauthorized (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
134822025-04-30T06:44:15.640ZTRACtest_unauthorized (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
134832025-04-30T06:44:15.640ZTRACtest_unauthorized (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
134842025-04-30T06:44:15.641ZINFOtest_unauthorized (dropshot): request completed kind = pantry latency_us = 8546 local_addr = [::1]:41451 method = GET remote_addr = [::1]:61826 req_id = f3d86888-6fbb-40a5-8cfb-ff4bf41a6c43 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
134852025-04-30T06:44:15.641ZINFOtest_unauthorized (dropshot): request completed kind = pantry latency_us = 8520 local_addr = [::1]:41451 method = GET remote_addr = [::1]:56138 req_id = 727e1c51-10e0-4c6e-bc5a-676bf3f5ae95 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
134862025-04-30T06:44:15.641ZINFOtest_unauthorized (dropshot): request completed kind = pantry latency_us = 8475 local_addr = [::1]:41451 method = GET remote_addr = [::1]:49375 req_id = 0a944a4a-9e3d-42b0-96da-f0d7005e54d5 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
134872025-04-30T06:44:15.644ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 2 table_name = fields_string
134882025-04-30T06:44:15.644ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
134892025-04-30T06:44:15.644ZDEBGtest_unauthorized (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 44264, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4320, tv_nsec: 443866583 } })
134902025-04-30T06:44:15.645ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
134912025-04-30T06:44:15.645ZDEBGtest_unauthorized (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 44264, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4320, tv_nsec: 444195438 } })
134922025-04-30T06:44:15.645ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
134932025-04-30T06:44:15.649ZINFOtest_unauthorized (dropshot_internal): request completed latency_us = 16228 local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 119604c6-3cbf-4995-83b7-bb670ee98bca response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
134942025-04-30T06:44:15.652ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 1 table_name = measurements_i64
134952025-04-30T06:44:15.652ZTRACtest_unauthorized (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
134962025-04-30T06:44:15.652ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 82 6b 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .k..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:52631 req_id = f2c9d45d-637b-4ab0-a87d-8c7384c15042
134972025-04-30T06:44:15.653ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 33387,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52631 req_id = f2c9d45d-637b-4ab0-a87d-8c7384c15042
134982025-04-30T06:44:15.653ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
134992025-04-30T06:44:15.653ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:52631 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = f2c9d45d-637b-4ab0-a87d-8c7384c15042
135002025-04-30T06:44:15.653ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: f7 99 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:63634 req_id = 1653afd7-b728-44d5-bca6-017328b64ab6
135012025-04-30T06:44:15.653ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 63385,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63634 req_id = 1653afd7-b728-44d5-bca6-017328b64ab6
135022025-04-30T06:44:15.653ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
135032025-04-30T06:44:15.653ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:63634 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 1653afd7-b728-44d5-bca6-017328b64ab6
135042025-04-30T06:44:15.653ZDEBGtest_unauthorized (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "119604c6-3cbf-4995-83b7-bb670ee98bca", "content-length": "405", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
135052025-04-30T06:44:15.653ZDEBGtest_unauthorized (oximeter): client request body = None method = GET uri = http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
135062025-04-30T06:44:15.654ZTRACtest_unauthorized (dropshot_internal): incoming request local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = f43bf33f-ed77-459c-bcf8-e68334e43cb7 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
135072025-04-30T06:44:15.655ZINFOtest_unauthorized (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager
135082025-04-30T06:44:15.711ZINFOtest_unauthorized (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager zone_address = ::1
135092025-04-30T06:44:15.711ZDEBGtest_unauthorized (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[::1]:12225/local/switch-id
135102025-04-30T06:44:15.711ZDEBGtest_unauthorized (MgsClient): client response background_task = bfd_manager result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
135112025-04-30T06:44:15.711ZWARNtest_unauthorized (BackgroundTasks): failed to identify switch slot for dendrite background_task = bfd_manager reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
135122025-04-30T06:44:15.712ZWARNtest_unauthorized (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = bfd_manager
135132025-04-30T06:44:15.713ZINFOtest_unauthorized (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc
135142025-04-30T06:44:15.789ZINFOtest_unauthorized (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc zone_address = ::1
135152025-04-30T06:44:15.789ZDEBGtest_unauthorized (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc uri = http://[::1]:12225/local/switch-id
135162025-04-30T06:44:15.789ZDEBGtest_unauthorized (MgsClient): client response background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
135172025-04-30T06:44:15.789ZWARNtest_unauthorized (BackgroundTasks): failed to identify switch slot for dendrite background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
135182025-04-30T06:44:15.793ZWARNtest_unauthorized (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc
135192025-04-30T06:44:15.799ZTRACtest_unauthorized (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = f43bf33f-ed77-459c-bcf8-e68334e43cb7 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
135202025-04-30T06:44:15.799ZDEBGtest_unauthorized (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = f43bf33f-ed77-459c-bcf8-e68334e43cb7 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
135212025-04-30T06:44:15.800ZDEBGtest_unauthorized (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = f43bf33f-ed77-459c-bcf8-e68334e43cb7 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
135222025-04-30T06:44:15.813ZWARNtest_unauthorized (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
135232025-04-30T06:44:15.820ZINFOtest_unauthorized (dropshot_internal): request completed latency_us = 165649 local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = f43bf33f-ed77-459c-bcf8-e68334e43cb7 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
135242025-04-30T06:44:15.823ZDEBGtest_unauthorized (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "f43bf33f-ed77-459c-bcf8-e68334e43cb7", "content-length": "29", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
135252025-04-30T06:44:15.823ZTRACtest_unauthorized (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 8d752adb-08b9-405f-a11b-66019bf447b5
135262025-04-30T06:44:15.823ZTRACtest_unauthorized (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
135272025-04-30T06:44:15.823ZINFOtest_unauthorized (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
135282025-04-30T06:44:15.836ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e8d23c98-efac-452b-a7bb-c6f12465677f", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
135292025-04-30T06:44:15.836ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
135302025-04-30T06:44:15.850ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
135312025-04-30T06:44:15.851ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0e9c1d60-ecd5-48d2-baf8-4c1a34e36e3a", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
135322025-04-30T06:44:15.851ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
135332025-04-30T06:44:15.853ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1bdd4d69-debc-47a3-ac40-5dea711b60d4", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
135342025-04-30T06:44:15.853ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
135352025-04-30T06:44:15.855ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "061cce6b-c399-4b4a-8542-0c50f0918c83", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
135362025-04-30T06:44:15.855ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
135372025-04-30T06:44:15.856ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6a3e231d-dd9c-44b6-a139-3dce0d05bb24", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
135382025-04-30T06:44:15.857ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
135392025-04-30T06:44:15.858ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f3c6fc7c-13a1-4465-bf1b-b1236f6462d4", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
135402025-04-30T06:44:15.858ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
135412025-04-30T06:44:15.859ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d35466b2-b1b7-4d4b-a078-d1b6f3c52091", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
135422025-04-30T06:44:15.860ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
135432025-04-30T06:44:15.861ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b13fa620-a806-4b75-a521-2ff2a52f1140", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
135442025-04-30T06:44:15.861ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
135452025-04-30T06:44:15.863ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "37ab2cad-53c9-4660-abc8-e901de1a7f97", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
135462025-04-30T06:44:15.863ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
135472025-04-30T06:44:15.864ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6d37351a-e1a5-4133-a38d-b34d3e806d2a", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
135482025-04-30T06:44:15.864ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
135492025-04-30T06:44:15.865ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d8d975ed-2394-4274-992f-e43072611773", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
135502025-04-30T06:44:15.865ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
135512025-04-30T06:44:15.867ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f08b5d08-f3d2-44e6-97fc-b790c387c2b6", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
135522025-04-30T06:44:15.867ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
135532025-04-30T06:44:15.869ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "371d977d-9518-405a-8172-cd13243f10a0", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
135542025-04-30T06:44:15.869ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
135552025-04-30T06:44:15.871ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a1b6c249-a985-4ec0-ab20-6adfec547b17", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
135562025-04-30T06:44:15.871ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
135572025-04-30T06:44:15.873ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b87e1e3f-91e6-43d1-a8c8-6f34f1d7ae67", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
135582025-04-30T06:44:15.873ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
135592025-04-30T06:44:15.875ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "698f0d1d-165b-4da5-861b-7236b5e5d5af", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
135602025-04-30T06:44:15.875ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
135612025-04-30T06:44:15.877ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "77f2f97b-5857-4aad-a428-457e67d3940b", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
135622025-04-30T06:44:15.877ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
135632025-04-30T06:44:15.878ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e4234a57-144e-4cc6-aab1-741b64c8196a", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
135642025-04-30T06:44:15.878ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
135652025-04-30T06:44:15.879ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3ffc5547-bc9a-4bbe-8ea5-a14d60d222c2", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
135662025-04-30T06:44:15.880ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
135672025-04-30T06:44:15.881ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2cc64bae-5972-4b1d-a28d-caf26c3d7cde", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
135682025-04-30T06:44:15.881ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
135692025-04-30T06:44:15.882ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cba30e43-98ef-4694-b4d2-9e1d4d424a32", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
135702025-04-30T06:44:15.882ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
135712025-04-30T06:44:15.884ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "db7dbea5-7d2d-45d6-8b62-3b554458e4c9", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
135722025-04-30T06:44:15.885ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
135732025-04-30T06:44:15.889ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "819b3a10-0d23-4899-9625-4cc3e129efeb", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
135742025-04-30T06:44:15.890ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
135752025-04-30T06:44:15.892ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e008d5e1-d1e5-4b1e-bde2-bae1c2bb750e", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
135762025-04-30T06:44:15.892ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
135772025-04-30T06:44:15.896ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b6975147-d603-4ebe-bbf4-444be8f822d0", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
135782025-04-30T06:44:15.896ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
135792025-04-30T06:44:15.899ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa0a7029-93bd-411a-845c-04a21f5574d1", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
135802025-04-30T06:44:15.899ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
135812025-04-30T06:44:15.901ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
135822025-04-30T06:44:15.901ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
135832025-04-30T06:44:15.901ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
135842025-04-30T06:44:15.901ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
135852025-04-30T06:44:15.901ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
135862025-04-30T06:44:15.903ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2be6a91d-3cc6-45b8-9090-37e85a3be851", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
135872025-04-30T06:44:15.903ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
135882025-04-30T06:44:15.904ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135892025-04-30T06:44:15.918ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
135902025-04-30T06:44:15.919ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
135912025-04-30T06:44:15.919ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
135922025-04-30T06:44:15.919ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
135932025-04-30T06:44:15.922ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
135942025-04-30T06:44:15.927ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
135952025-04-30T06:44:15.928ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
135962025-04-30T06:44:15.930ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "eb26c867-a2ed-4fb2-beec-3f7bd461628f", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
135972025-04-30T06:44:15.930ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
135982025-04-30T06:44:15.931ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5120420c-e3ac-4f44-964f-f38ee1a3af7c", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
135992025-04-30T06:44:15.932ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
136002025-04-30T06:44:15.933ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4e8ac552-7825-4793-bc2f-0dbede48b1f2", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
136012025-04-30T06:44:15.933ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
136022025-04-30T06:44:15.936ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b66aaf11-7a8d-4f73-a895-c3ae7007eb54", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
136032025-04-30T06:44:15.936ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
136042025-04-30T06:44:15.938ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c629d793-664e-4631-a3cc-294eded2d97e", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
136052025-04-30T06:44:15.939ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
136062025-04-30T06:44:15.942ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "18ec1474-7f7d-424f-932c-3693b88b9626", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
136072025-04-30T06:44:15.942ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
136082025-04-30T06:44:15.945ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "123715dd-5351-450f-b2cc-45386e9e627d", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
136092025-04-30T06:44:15.945ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
136102025-04-30T06:44:15.949ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3253d99f-d418-450d-aa5a-77ceb37714ed", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
136112025-04-30T06:44:15.949ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
136122025-04-30T06:44:15.954ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "62042e68-1553-475a-a074-870bdfb0575a", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
136132025-04-30T06:44:15.954ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
136142025-04-30T06:44:15.961ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
136152025-04-30T06:44:15.964ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a6e360a6-0bb0-4641-907f-ec2509681987", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
136162025-04-30T06:44:15.964ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
136172025-04-30T06:44:15.968ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "715de84c-2baa-492a-82b3-e890c2312a72", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
136182025-04-30T06:44:15.968ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
136192025-04-30T06:44:15.971ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2fed6977-4489-47d9-bc9e-df5215b99bb9", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
136202025-04-30T06:44:15.971ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
136212025-04-30T06:44:15.972ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "120086ed-61a7-41b7-bedb-f9019341bd07", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
136222025-04-30T06:44:15.973ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
136232025-04-30T06:44:15.974ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "30f023b0-4c72-456a-b25a-ac6475795486", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
136242025-04-30T06:44:15.980ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
136252025-04-30T06:44:15.982ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a4a631cc-2dc4-482b-8466-10431da811bd", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
136262025-04-30T06:44:15.983ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
136272025-04-30T06:44:15.985ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2bf8751a-53a7-42e1-a126-f672952b7e1c", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
136282025-04-30T06:44:15.985ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
136292025-04-30T06:44:15.986ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
136302025-04-30T06:44:15.986ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
136312025-04-30T06:44:15.986ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
136322025-04-30T06:44:15.987ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
136332025-04-30T06:44:15.989ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136342025-04-30T06:44:15.993ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
136352025-04-30T06:44:15.993ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
136362025-04-30T06:44:15.993ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
136372025-04-30T06:44:15.993ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
136382025-04-30T06:44:15.993ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
136392025-04-30T06:44:15.994ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
136402025-04-30T06:44:15.997ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136412025-04-30T06:44:16.008ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
136422025-04-30T06:44:16.008ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
136432025-04-30T06:44:16.008ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
136442025-04-30T06:44:16.009ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
136452025-04-30T06:44:16.012ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
136462025-04-30T06:44:16.012ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
136472025-04-30T06:44:16.012ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
136482025-04-30T06:44:16.012ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
136492025-04-30T06:44:16.013ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
136502025-04-30T06:44:16.018ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136512025-04-30T06:44:16.028ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
136522025-04-30T06:44:16.028ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
136532025-04-30T06:44:16.028ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
136542025-04-30T06:44:16.030ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
136552025-04-30T06:44:16.090ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
136562025-04-30T06:44:16.090ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory
136572025-04-30T06:44:16.091ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:38434 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
136582025-04-30T06:44:16.091ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:38434 req_id = bb6d424a-3ae3-4450-834d-265d080fb1ea sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
136592025-04-30T06:44:16.093ZINFOtest_unauthorized (dropshot): request completed latency_us = 1492 local_addr = [::1]:57786 method = GET remote_addr = [::1]:38434 req_id = bb6d424a-3ae3-4450-834d-265d080fb1ea response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
136602025-04-30T06:44:16.093ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
136612025-04-30T06:44:16.093ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
136622025-04-30T06:44:16.094ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
136632025-04-30T06:44:16.095ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = http://[::1]:57786 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:57786/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb6d424a-3ae3-4450-834d-265d080fb1ea", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
136642025-04-30T06:44:16.097ZDEBGtest_unauthorized (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
136652025-04-30T06:44:16.097ZDEBGtest_unauthorized (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
136662025-04-30T06:44:16.097ZDEBGtest_unauthorized (BackgroundTasks): finished collection background_task = inventory_collection
136672025-04-30T06:44:16.099ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
136682025-04-30T06:44:16.099ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
136692025-04-30T06:44:16.099ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
136702025-04-30T06:44:16.099ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
136712025-04-30T06:44:16.100ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
136722025-04-30T06:44:16.110ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136732025-04-30T06:44:16.117ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
136742025-04-30T06:44:16.117ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
136752025-04-30T06:44:16.117ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
136762025-04-30T06:44:16.118ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
136772025-04-30T06:44:16.167ZWARNtest_unauthorized (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
136782025-04-30T06:44:16.168ZWARNtest_unauthorized (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
136792025-04-30T06:44:16.168ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 1.297438262s iteration = 49 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
136802025-04-30T06:44:16.168ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 50 reason = Signaled
136812025-04-30T06:44:16.168ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
136822025-04-30T06:44:16.168ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
136832025-04-30T06:44:16.168ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
136842025-04-30T06:44:16.168ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
136852025-04-30T06:44:16.169ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
136862025-04-30T06:44:16.170ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
136872025-04-30T06:44:16.172ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136882025-04-30T06:44:16.177ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
136892025-04-30T06:44:16.177ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
136902025-04-30T06:44:16.177ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
136912025-04-30T06:44:16.179ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
136922025-04-30T06:44:16.179ZDEBGtest_unauthorized (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
136932025-04-30T06:44:16.179ZDEBGtest_unauthorized (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
136942025-04-30T06:44:16.180ZDEBGtest_unauthorized (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a
136952025-04-30T06:44:16.182ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
136962025-04-30T06:44:16.186ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1a8b2318-abf8-406d-a389-112ab91a0de8, 0)]
136972025-04-30T06:44:16.186ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
136982025-04-30T06:44:16.190ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 2 table_name = fields_string
136992025-04-30T06:44:16.190ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
137002025-04-30T06:44:16.193ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: b5 67 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .g..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:58868 req_id = c7eb1f98-fa2a-4fdd-92f7-4291e20ca561
137012025-04-30T06:44:16.193ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 46439,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:58868 req_id = c7eb1f98-fa2a-4fdd-92f7-4291e20ca561
137022025-04-30T06:44:16.193ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
137032025-04-30T06:44:16.193ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
137042025-04-30T06:44:16.194ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:58868 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = c7eb1f98-fa2a-4fdd-92f7-4291e20ca561
137052025-04-30T06:44:16.194ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4320, tv_nsec: 993018763 } })
137062025-04-30T06:44:16.194ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
137072025-04-30T06:44:16.194ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 1 table_name = measurements_i64
137082025-04-30T06:44:16.194ZTRACtest_unauthorized (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
137092025-04-30T06:44:16.196ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 07 de 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:60801 req_id = 40329d3e-0c99-48d9-a470-ad9a3abbb8f3
137102025-04-30T06:44:16.196ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 2014,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60801 req_id = 40329d3e-0c99-48d9-a470-ad9a3abbb8f3
137112025-04-30T06:44:16.196ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
137122025-04-30T06:44:16.196ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:60801 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 40329d3e-0c99-48d9-a470-ad9a3abbb8f3
137132025-04-30T06:44:16.286ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
137142025-04-30T06:44:16.287ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
137152025-04-30T06:44:16.287ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
137162025-04-30T06:44:16.287ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
137172025-04-30T06:44:16.287ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
137182025-04-30T06:44:16.288ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
137192025-04-30T06:44:16.288ZDEBGtest_unauthorized (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
137202025-04-30T06:44:16.288ZDEBGtest_unauthorized (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
137212025-04-30T06:44:16.289ZWARNtest_unauthorized (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
137222025-04-30T06:44:16.291ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 32 62 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c 2b..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:61669 req_id = 9a24a7a1-270b-47b3-9705-d7df91624e61
137232025-04-30T06:44:16.291ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 12898,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:61669 req_id = 9a24a7a1-270b-47b3-9705-d7df91624e61
137242025-04-30T06:44:16.291ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
137252025-04-30T06:44:16.291ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 12898, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:61669 req_id = 9a24a7a1-270b-47b3-9705-d7df91624e61
137262025-04-30T06:44:16.294ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
137272025-04-30T06:44:16.294ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
137282025-04-30T06:44:16.294ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
137292025-04-30T06:44:16.295ZTRACtest_unauthorized (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
137302025-04-30T06:44:16.296ZDEBGtest_unauthorized (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
137312025-04-30T06:44:16.296ZTRACtest_unauthorized (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
137322025-04-30T06:44:16.296ZTRACtest_unauthorized (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
137332025-04-30T06:44:16.297ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137342025-04-30T06:44:16.302ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
137352025-04-30T06:44:16.302ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
137362025-04-30T06:44:16.302ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
137372025-04-30T06:44:16.303ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
137382025-04-30T06:44:16.306ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "c6778f06-6650-4be6-9d88-97a3f4172e0e", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
137392025-04-30T06:44:16.307ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
137402025-04-30T06:44:16.307ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
137412025-04-30T06:44:16.307ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
137422025-04-30T06:44:16.307ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
137432025-04-30T06:44:16.309ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cf03d10e-bbc8-449c-971f-06d2c0e80594", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
137442025-04-30T06:44:16.309ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
137452025-04-30T06:44:16.311ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "89fac678-035a-4380-acbe-6738fb196120", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
137462025-04-30T06:44:16.312ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
137472025-04-30T06:44:16.314ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2239760f-12b6-445b-86bb-0d4e7182ebbf", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
137482025-04-30T06:44:16.314ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
137492025-04-30T06:44:16.319ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1792165b-c78e-47ae-be1a-a1c8e7083d90", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
137502025-04-30T06:44:16.319ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
137512025-04-30T06:44:16.321ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "febdd46a-1635-450d-9de1-c5f0ef498573", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
137522025-04-30T06:44:16.322ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
137532025-04-30T06:44:16.324ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e0f29d99-e4a4-4d52-9695-e4a31ed82b8f", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
137542025-04-30T06:44:16.324ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
137552025-04-30T06:44:16.327ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9438845f-7550-4fb3-b7f0-b99eadb5fddf", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
137562025-04-30T06:44:16.327ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
137572025-04-30T06:44:16.329ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "320268e0-1d01-498d-b780-067e169b6e9c", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
137582025-04-30T06:44:16.329ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
137592025-04-30T06:44:16.329ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "de1f42d6-1593-40cd-8098-f2abf1d4dadd", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
137602025-04-30T06:44:16.330ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
137612025-04-30T06:44:16.331ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "754544e7-4c97-422b-a690-675f15717d37", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
137622025-04-30T06:44:16.331ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
137632025-04-30T06:44:16.333ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6023a4a8-0bc8-4902-985e-037de8aa0312", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
137642025-04-30T06:44:16.333ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
137652025-04-30T06:44:16.335ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb7378c2-ec45-4b39-b958-cba55d20c31d", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
137662025-04-30T06:44:16.335ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
137672025-04-30T06:44:16.339ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
137682025-04-30T06:44:16.340ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fe26b06a-f95b-462d-8aa9-0a291b6eb7df", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
137692025-04-30T06:44:16.340ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
137702025-04-30T06:44:16.342ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ff2b461b-3bc9-43d2-a617-9539177a0725", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
137712025-04-30T06:44:16.342ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
137722025-04-30T06:44:16.345ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ef73d85c-8446-4772-bd79-a72272832719", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
137732025-04-30T06:44:16.345ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
137742025-04-30T06:44:16.347ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0884a063-a76f-4269-8b1c-9d627aec20eb", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
137752025-04-30T06:44:16.347ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
137762025-04-30T06:44:16.351ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "93d47a3c-4c0f-4818-becb-75ddfc61fde5", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
137772025-04-30T06:44:16.351ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
137782025-04-30T06:44:16.353ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7c3da34a-ec99-447f-84fc-a89fd4a0477e", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
137792025-04-30T06:44:16.353ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
137802025-04-30T06:44:16.355ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e1d43cf3-97fd-4186-98c3-8f73cacdb591", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
137812025-04-30T06:44:16.355ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
137822025-04-30T06:44:16.357ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "55443e84-e4c8-4b01-8e25-ddc8f5adb568", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
137832025-04-30T06:44:16.357ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
137842025-04-30T06:44:16.359ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "002ed44c-6563-4c82-acc6-fedfa2cd68eb", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
137852025-04-30T06:44:16.359ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
137862025-04-30T06:44:16.361ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fba16937-16ef-4fe9-890c-3f8da9fb7e58", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
137872025-04-30T06:44:16.361ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
137882025-04-30T06:44:16.365ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "85cbb1f5-8090-4a20-9668-898cbaed8d25", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
137892025-04-30T06:44:16.366ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
137902025-04-30T06:44:16.368ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b059e4e5-88e9-4eac-8bbd-0bedd04f2b11", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
137912025-04-30T06:44:16.368ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
137922025-04-30T06:44:16.371ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5c5982a2-0a59-4f25-83f5-8c8f31636427", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
137932025-04-30T06:44:16.371ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
137942025-04-30T06:44:16.375ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7c3ee214-ad1e-421a-833e-351e8a7746aa", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
137952025-04-30T06:44:16.375ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
137962025-04-30T06:44:16.378ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "46aa326b-51aa-4e3a-bf08-a860a9cbba08", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
137972025-04-30T06:44:16.378ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
137982025-04-30T06:44:16.384ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "feb304fa-00f6-4e8c-8b9a-aa91552b2136", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
137992025-04-30T06:44:16.384ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
138002025-04-30T06:44:16.388ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f6ce86b5-860d-4232-9693-111eb5a18df9", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
138012025-04-30T06:44:16.388ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
138022025-04-30T06:44:16.390ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "58ad3b96-5f92-4038-a38d-189a2421e915", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
138032025-04-30T06:44:16.390ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
138042025-04-30T06:44:16.390ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
138052025-04-30T06:44:16.390ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
138062025-04-30T06:44:16.390ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
138072025-04-30T06:44:16.390ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
138082025-04-30T06:44:16.391ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
138092025-04-30T06:44:16.392ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b680d84f-e1b1-4b1a-855a-3086bf9f499f", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
138102025-04-30T06:44:16.392ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
138112025-04-30T06:44:16.395ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138122025-04-30T06:44:16.403ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
138132025-04-30T06:44:16.403ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
138142025-04-30T06:44:16.403ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
138152025-04-30T06:44:16.404ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
138162025-04-30T06:44:16.404ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9e1ae9bb-3636-4ccb-9fc5-e3a36313db6d", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
138172025-04-30T06:44:16.404ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
138182025-04-30T06:44:16.406ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "30c96d0f-9659-4d90-8c4f-a61c83492eca", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
138192025-04-30T06:44:16.406ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
138202025-04-30T06:44:16.407ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
138212025-04-30T06:44:16.407ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
138222025-04-30T06:44:16.407ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
138232025-04-30T06:44:16.410ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7411ff10-0a14-4bee-b869-95d093743d31", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
138242025-04-30T06:44:16.410ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
138252025-04-30T06:44:16.414ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c3ce95fa-9b5c-4580-9407-ae6615f4c8ff", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
138262025-04-30T06:44:16.414ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
138272025-04-30T06:44:16.426ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "23bab49c-9276-4606-ab5d-d2042bfc7cb7", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
138282025-04-30T06:44:16.426ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
138292025-04-30T06:44:16.437ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "eaabda43-e274-4d0a-ac68-1172008d02c1", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
138302025-04-30T06:44:16.437ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
138312025-04-30T06:44:16.443ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "dc8cc3d0-d38d-4eef-a4ba-6cce6bc53b7e", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
138322025-04-30T06:44:16.443ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
138332025-04-30T06:44:16.448ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0197b73c-2673-4b93-953f-98215c163c33", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
138342025-04-30T06:44:16.450ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
138352025-04-30T06:44:16.456ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
138362025-04-30T06:44:16.459ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0211e76b-fcc6-4cba-9c44-04b1a60ac175", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
138372025-04-30T06:44:16.459ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
138382025-04-30T06:44:16.460ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a9ca795-3015-4364-9c18-dd9553bc77b8", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
138392025-04-30T06:44:16.460ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
138402025-04-30T06:44:16.462ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a480e5f4-1209-4cf9-97ed-2a8ca581081d", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
138412025-04-30T06:44:16.462ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
138422025-04-30T06:44:16.464ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ba662834-1039-4d2e-8a93-adb0742e2a53", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
138432025-04-30T06:44:16.464ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
138442025-04-30T06:44:16.467ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6720ccb5-75a9-48e3-b852-3f8b97f722f8", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
138452025-04-30T06:44:16.467ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
138462025-04-30T06:44:16.467ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
138472025-04-30T06:44:16.467ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
138482025-04-30T06:44:16.468ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
138492025-04-30T06:44:16.470ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
138502025-04-30T06:44:16.475ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138512025-04-30T06:44:16.483ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
138522025-04-30T06:44:16.483ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
138532025-04-30T06:44:16.483ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
138542025-04-30T06:44:16.483ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
138552025-04-30T06:44:16.483ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
138562025-04-30T06:44:16.484ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
138572025-04-30T06:44:16.489ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138582025-04-30T06:44:16.494ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
138592025-04-30T06:44:16.494ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
138602025-04-30T06:44:16.494ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
138612025-04-30T06:44:16.495ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
138622025-04-30T06:44:16.556ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
138632025-04-30T06:44:16.557ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory
138642025-04-30T06:44:16.558ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
138652025-04-30T06:44:16.558ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
138662025-04-30T06:44:16.558ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
138672025-04-30T06:44:16.559ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
138682025-04-30T06:44:16.560ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
138692025-04-30T06:44:16.560ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:58170 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
138702025-04-30T06:44:16.564ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:58170 req_id = 3d48c9c0-3f14-40dd-935f-182451ac75ef sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
138712025-04-30T06:44:16.565ZINFOtest_unauthorized (dropshot): request completed latency_us = 1259 local_addr = [::1]:57786 method = GET remote_addr = [::1]:58170 req_id = 3d48c9c0-3f14-40dd-935f-182451ac75ef response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
138722025-04-30T06:44:16.565ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138732025-04-30T06:44:16.574ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
138742025-04-30T06:44:16.574ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
138752025-04-30T06:44:16.575ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
138762025-04-30T06:44:16.575ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
138772025-04-30T06:44:16.576ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = http://[::1]:57786 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:57786/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "3d48c9c0-3f14-40dd-935f-182451ac75ef", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
138782025-04-30T06:44:16.580ZDEBGtest_unauthorized (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
138792025-04-30T06:44:16.581ZDEBGtest_unauthorized (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
138802025-04-30T06:44:16.581ZDEBGtest_unauthorized (BackgroundTasks): finished collection background_task = inventory_collection
138812025-04-30T06:44:16.583ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
138822025-04-30T06:44:16.583ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
138832025-04-30T06:44:16.583ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
138842025-04-30T06:44:16.583ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
138852025-04-30T06:44:16.584ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
138862025-04-30T06:44:16.589ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
138872025-04-30T06:44:16.589ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
138882025-04-30T06:44:16.591ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
138892025-04-30T06:44:16.591ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138902025-04-30T06:44:16.595ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
138912025-04-30T06:44:16.595ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
138922025-04-30T06:44:16.595ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
138932025-04-30T06:44:16.597ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
138942025-04-30T06:44:16.670ZWARNtest_unauthorized (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
138952025-04-30T06:44:16.671ZWARNtest_unauthorized (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
138962025-04-30T06:44:16.671ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 503.470903ms iteration = 50 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
138972025-04-30T06:44:16.671ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 51 reason = Signaled
138982025-04-30T06:44:16.671ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
138992025-04-30T06:44:16.674ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
139002025-04-30T06:44:16.674ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
139012025-04-30T06:44:16.674ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
139022025-04-30T06:44:16.675ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
139032025-04-30T06:44:16.679ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
139042025-04-30T06:44:16.682ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139052025-04-30T06:44:16.688ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
139062025-04-30T06:44:16.688ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
139072025-04-30T06:44:16.688ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
139082025-04-30T06:44:16.688ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
139092025-04-30T06:44:16.692ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1a8b2318-abf8-406d-a389-112ab91a0de8, 0)]
139102025-04-30T06:44:16.692ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
139112025-04-30T06:44:16.693ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: bf 2b 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .+..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:62970 req_id = bbb880a1-ebfc-4d44-a306-f63a5bbb3576
139122025-04-30T06:44:16.693ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 48939,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62970 req_id = bbb880a1-ebfc-4d44-a306-f63a5bbb3576
139132025-04-30T06:44:16.693ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
139142025-04-30T06:44:16.693ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
139152025-04-30T06:44:16.693ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:62970 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = bbb880a1-ebfc-4d44-a306-f63a5bbb3576
139162025-04-30T06:44:16.693ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4321, tv_nsec: 492766408 } })
139172025-04-30T06:44:16.693ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
139182025-04-30T06:44:16.694ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: e2 6c 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .l..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:49181 req_id = a6531ad9-289e-440e-8678-ac5d9e7390c7
139192025-04-30T06:44:16.694ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 57964,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:49181 req_id = a6531ad9-289e-440e-8678-ac5d9e7390c7
139202025-04-30T06:44:16.694ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
139212025-04-30T06:44:16.694ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:49181 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = a6531ad9-289e-440e-8678-ac5d9e7390c7
139222025-04-30T06:44:16.763ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
139232025-04-30T06:44:16.763ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
139242025-04-30T06:44:16.763ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
139252025-04-30T06:44:16.763ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
139262025-04-30T06:44:16.763ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
139272025-04-30T06:44:16.767ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
139282025-04-30T06:44:16.767ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 58 bf 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c X............_cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:56884 req_id = c02e4def-5797-4bef-9e6e-208205ffa270
139292025-04-30T06:44:16.768ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 22719,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:56884 req_id = c02e4def-5797-4bef-9e6e-208205ffa270
139302025-04-30T06:44:16.768ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
139312025-04-30T06:44:16.769ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 22719, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:56884 req_id = c02e4def-5797-4bef-9e6e-208205ffa270
139322025-04-30T06:44:16.769ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
139332025-04-30T06:44:16.769ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
139342025-04-30T06:44:16.769ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
139352025-04-30T06:44:16.771ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139362025-04-30T06:44:16.783ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
139372025-04-30T06:44:16.783ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
139382025-04-30T06:44:16.783ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
139392025-04-30T06:44:16.783ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
139402025-04-30T06:44:16.786ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "2ab58e61-1460-4a01-a388-61ca67d883e2", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
139412025-04-30T06:44:16.786ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
139422025-04-30T06:44:16.787ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
139432025-04-30T06:44:16.787ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
139442025-04-30T06:44:16.787ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
139452025-04-30T06:44:16.790ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cee3bcf2-1c7c-4653-b47f-cdb290e704c6", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
139462025-04-30T06:44:16.791ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
139472025-04-30T06:44:16.793ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "49f8dd1c-758b-4c6e-ad30-6b1ddfdff579", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
139482025-04-30T06:44:16.793ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
139492025-04-30T06:44:16.798ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "730adbaa-c458-48b2-9acc-8a6b56f2ba75", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
139502025-04-30T06:44:16.798ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
139512025-04-30T06:44:16.803ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "aeba3de8-197e-41db-928f-1fbde92a66df", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
139522025-04-30T06:44:16.803ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
139532025-04-30T06:44:16.805ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ea9c8b83-5477-4616-8b16-2b12be84df9b", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
139542025-04-30T06:44:16.805ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
139552025-04-30T06:44:16.809ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3cd6aefa-9e30-45a5-bdc9-b6e5c1244074", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
139562025-04-30T06:44:16.809ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
139572025-04-30T06:44:16.816ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "976cd2db-6065-4ad9-a01f-4e579426a9da", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
139582025-04-30T06:44:16.816ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
139592025-04-30T06:44:16.817ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6b707afb-fd23-49f7-8a6a-1c65e026258c", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
139602025-04-30T06:44:16.817ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
139612025-04-30T06:44:16.818ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4577364d-e90e-48bb-b767-592bf0f9551f", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
139622025-04-30T06:44:16.818ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
139632025-04-30T06:44:16.820ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "72a8838b-560c-4dd5-8575-e1ab618413bb", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
139642025-04-30T06:44:16.820ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
139652025-04-30T06:44:16.840ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ba1ff398-3dfd-43d9-a01e-a0da683b9279", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
139662025-04-30T06:44:16.840ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
139672025-04-30T06:44:16.841ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
139682025-04-30T06:44:16.842ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5cc8ac36-b4ba-4856-8eb5-2ebe46139111", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
139692025-04-30T06:44:16.843ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
139702025-04-30T06:44:16.844ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "625ac133-2b0b-43f3-9ba1-77ed775aef2b", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
139712025-04-30T06:44:16.844ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
139722025-04-30T06:44:16.846ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e8603d5f-1737-47d5-88c7-1c6451279fbf", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
139732025-04-30T06:44:16.854ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
139742025-04-30T06:44:16.858ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d10dd2b2-9033-4cd6-9343-17e94d16ca0b", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
139752025-04-30T06:44:16.858ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
139762025-04-30T06:44:16.860ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "508683cc-55c0-45cc-8547-b63dfe24477a", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
139772025-04-30T06:44:16.860ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
139782025-04-30T06:44:16.862ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3c554317-9135-4328-9e06-a565d00c0628", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
139792025-04-30T06:44:16.863ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
139802025-04-30T06:44:16.865ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6740a362-887d-4a2d-ab4a-72cd783cc87a", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
139812025-04-30T06:44:16.865ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
139822025-04-30T06:44:16.866ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a79f6024-f495-44d4-884b-0e9aadb0b0ac", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
139832025-04-30T06:44:16.866ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
139842025-04-30T06:44:16.867ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "46192e5f-939e-478d-81c4-e08e237c077b", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
139852025-04-30T06:44:16.868ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
139862025-04-30T06:44:16.869ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8480e58d-82a4-4a03-b294-bae57d824244", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
139872025-04-30T06:44:16.869ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
139882025-04-30T06:44:16.872ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3572382f-2ba4-408d-b399-01285a6113d5", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
139892025-04-30T06:44:16.872ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
139902025-04-30T06:44:16.873ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "be9bbc98-bfb0-4ee7-8d95-6db8f31d80a3", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
139912025-04-30T06:44:16.873ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
139922025-04-30T06:44:16.875ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "49354b1d-5c00-4524-acfe-8a8082471148", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
139932025-04-30T06:44:16.875ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
139942025-04-30T06:44:16.878ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a8171d81-6af6-4fea-b028-2e8bbdcc2310", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
139952025-04-30T06:44:16.878ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
139962025-04-30T06:44:16.881ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0dcef88b-10a2-44d6-bd64-475f8d958e35", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
139972025-04-30T06:44:16.881ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
139982025-04-30T06:44:16.883ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1b74f73e-37c8-4baa-ba8e-df21a04b2716", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
139992025-04-30T06:44:16.883ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
140002025-04-30T06:44:16.885ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9789ca35-d9de-4930-ad03-30f7cc7e81e7", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
140012025-04-30T06:44:16.885ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
140022025-04-30T06:44:16.888ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a3dcf54f-d409-47de-bfed-46a2211986e4", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
140032025-04-30T06:44:16.888ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
140042025-04-30T06:44:16.890ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "02607d60-c0f5-43b7-8472-44da786e64b0", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
140052025-04-30T06:44:16.890ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
140062025-04-30T06:44:16.890ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5b61de43-ed7a-4165-af3c-a6b83f0077d8", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
140072025-04-30T06:44:16.890ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
140082025-04-30T06:44:16.892ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
140092025-04-30T06:44:16.892ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
140102025-04-30T06:44:16.892ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
140112025-04-30T06:44:16.892ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
140122025-04-30T06:44:16.893ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
140132025-04-30T06:44:16.894ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0e89174d-8c5d-430a-a0c7-f605dda2af1c", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
140142025-04-30T06:44:16.894ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
140152025-04-30T06:44:16.896ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
140162025-04-30T06:44:16.902ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
140172025-04-30T06:44:16.902ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
140182025-04-30T06:44:16.902ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
140192025-04-30T06:44:16.903ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
140202025-04-30T06:44:16.904ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cae556af-244a-42e7-87c3-b7f268201a86", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
140212025-04-30T06:44:16.904ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
140222025-04-30T06:44:16.906ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5245a881-fd0c-4353-8e85-e348dd88fcb8", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
140232025-04-30T06:44:16.907ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
140242025-04-30T06:44:16.907ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
140252025-04-30T06:44:16.907ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
140262025-04-30T06:44:16.907ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
140272025-04-30T06:44:16.911ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e01e4af6-65de-40f3-90da-50b162e8ec49", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
140282025-04-30T06:44:16.911ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
140292025-04-30T06:44:16.914ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1f793da8-0cb2-4263-a052-ee611eb35c24", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
140302025-04-30T06:44:16.914ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
140312025-04-30T06:44:16.917ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9b065f1b-462f-4965-97cd-36b8ae1a73f4", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
140322025-04-30T06:44:16.917ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
140332025-04-30T06:44:16.921ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e1f3ba34-3226-405a-bd4a-c24111374a8f", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
140342025-04-30T06:44:16.922ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
140352025-04-30T06:44:16.926ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a52c09ee-86a3-49f1-81c0-ad95c7520962", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
140362025-04-30T06:44:16.926ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
140372025-04-30T06:44:16.928ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d0e1ccdd-b36f-43da-b501-bf7445f71ad3", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
140382025-04-30T06:44:16.928ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
140392025-04-30T06:44:16.930ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "90af9045-07e6-4303-b9bf-305eef258c06", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
140402025-04-30T06:44:16.930ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
140412025-04-30T06:44:16.931ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "846bc6b9-daaa-4c41-ab09-2f7e35c15ca9", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
140422025-04-30T06:44:16.931ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
140432025-04-30T06:44:16.933ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "11ea6725-ff5a-4c8d-83d9-44c7aaf62097", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
140442025-04-30T06:44:16.933ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
140452025-04-30T06:44:16.935ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "bdd89362-11f1-483d-b421-219fb330654d", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
140462025-04-30T06:44:16.935ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
140472025-04-30T06:44:16.935ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
140482025-04-30T06:44:16.935ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
140492025-04-30T06:44:16.936ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
140502025-04-30T06:44:16.937ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
140512025-04-30T06:44:16.942ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
140522025-04-30T06:44:16.950ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
140532025-04-30T06:44:16.950ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
140542025-04-30T06:44:16.950ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
140552025-04-30T06:44:16.950ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
140562025-04-30T06:44:16.950ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
140572025-04-30T06:44:16.952ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
140582025-04-30T06:44:16.953ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
140592025-04-30T06:44:16.955ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
140602025-04-30T06:44:16.961ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
140612025-04-30T06:44:16.961ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
140622025-04-30T06:44:16.961ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
140632025-04-30T06:44:16.962ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
140642025-04-30T06:44:17.035ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
140652025-04-30T06:44:17.035ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory
140662025-04-30T06:44:17.036ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
140672025-04-30T06:44:17.036ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
140682025-04-30T06:44:17.036ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
140692025-04-30T06:44:17.036ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
140702025-04-30T06:44:17.038ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
140712025-04-30T06:44:17.038ZINFOtest_unauthorized (dropshot): accepted connection local_addr = [::1]:57786 remote_addr = [::1]:49737 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
140722025-04-30T06:44:17.039ZTRACtest_unauthorized (dropshot): incoming request local_addr = [::1]:57786 method = GET remote_addr = [::1]:49737 req_id = 3838bed8-48f8-48ac-bd41-8357915f70d6 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
140732025-04-30T06:44:17.041ZINFOtest_unauthorized (dropshot): request completed latency_us = 2222 local_addr = [::1]:57786 method = GET remote_addr = [::1]:49737 req_id = 3838bed8-48f8-48ac-bd41-8357915f70d6 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
140742025-04-30T06:44:17.042ZDEBGtest_unauthorized (BackgroundTasks): client response SledAgent = http://[::1]:57786 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:57786/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "3838bed8-48f8-48ac-bd41-8357915f70d6", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
140752025-04-30T06:44:17.047ZDEBGtest_unauthorized (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
140762025-04-30T06:44:17.048ZDEBGtest_unauthorized (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
140772025-04-30T06:44:17.048ZDEBGtest_unauthorized (BackgroundTasks): finished collection background_task = inventory_collection
140782025-04-30T06:44:17.051ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
140792025-04-30T06:44:17.051ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
140802025-04-30T06:44:17.051ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
140812025-04-30T06:44:17.051ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
140822025-04-30T06:44:17.052ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
140832025-04-30T06:44:17.053ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
140842025-04-30T06:44:17.061ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
140852025-04-30T06:44:17.061ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
140862025-04-30T06:44:17.061ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
140872025-04-30T06:44:17.062ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
140882025-04-30T06:44:17.065ZTRACtest_unauthorized (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
140892025-04-30T06:44:17.066ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
140902025-04-30T06:44:17.071ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
140912025-04-30T06:44:17.072ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
140922025-04-30T06:44:17.072ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
140932025-04-30T06:44:17.072ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
140942025-04-30T06:44:17.074ZTRACtest_unauthorized (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 40 54 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 @T..........._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:59665 req_id = f6f6b715-8b2f-40e0-97fd-643c97794ac0
140952025-04-30T06:44:17.074ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 16468,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:59665 req_id = f6f6b715-8b2f-40e0-97fd-643c97794ac0
140962025-04-30T06:44:17.074ZDEBGtest_unauthorized (store): query key key = _dendrite._tcp
140972025-04-30T06:44:17.074ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
140982025-04-30T06:44:17.074ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:59665 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 44264, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = f6f6b715-8b2f-40e0-97fd-643c97794ac0
140992025-04-30T06:44:17.075ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
141002025-04-30T06:44:17.075ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
141012025-04-30T06:44:17.076ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
141022025-04-30T06:44:17.077ZDEBGtest_unauthorized (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 44264, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4321, tv_nsec: 876235875 } })
141032025-04-30T06:44:17.077ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
141042025-04-30T06:44:17.078ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: fb e2 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:58810 req_id = 70bf7273-d81a-44b7-b7a1-dbcf45d01bc1
141052025-04-30T06:44:17.078ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 64482,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:58810 req_id = 70bf7273-d81a-44b7-b7a1-dbcf45d01bc1
141062025-04-30T06:44:17.078ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
141072025-04-30T06:44:17.078ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:58810 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 70bf7273-d81a-44b7-b7a1-dbcf45d01bc1
141082025-04-30T06:44:17.079ZINFOtest_unauthorized (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector
141092025-04-30T06:44:17.138ZINFOtest_unauthorized (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector zone_address = ::1
141102025-04-30T06:44:17.138ZDEBGtest_unauthorized (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[::1]:12225/local/switch-id
141112025-04-30T06:44:17.138ZDEBGtest_unauthorized (MgsClient): client response background_task = nat_v4_garbage_collector result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
141122025-04-30T06:44:17.138ZWARNtest_unauthorized (BackgroundTasks): failed to identify switch slot for dendrite background_task = nat_v4_garbage_collector reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
141132025-04-30T06:44:17.141ZWARNtest_unauthorized (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = nat_v4_garbage_collector
141142025-04-30T06:44:17.142ZINFOtest_unauthorized (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
141152025-04-30T06:44:17.142ZDEBGtest_unauthorized (oximeter): client request body = None method = GET uri = http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
141162025-04-30T06:44:17.144ZTRACtest_unauthorized (dropshot_internal): incoming request local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 2d7b53f5-4423-4647-94d7-26f50f885c8d uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
141172025-04-30T06:44:17.144ZTRACtest_unauthorized (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 2d7b53f5-4423-4647-94d7-26f50f885c8d resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
141182025-04-30T06:44:17.144ZDEBGtest_unauthorized (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 2d7b53f5-4423-4647-94d7-26f50f885c8d roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
141192025-04-30T06:44:17.145ZDEBGtest_unauthorized (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 2d7b53f5-4423-4647-94d7-26f50f885c8d resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
141202025-04-30T06:44:17.157ZINFOtest_unauthorized (dropshot_internal): request completed latency_us = 13221 local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 2d7b53f5-4423-4647-94d7-26f50f885c8d response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
141212025-04-30T06:44:17.159ZDEBGtest_unauthorized (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "2d7b53f5-4423-4647-94d7-26f50f885c8d", "content-length": "405", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
141222025-04-30T06:44:17.160ZDEBGtest_unauthorized (oximeter): client request body = None method = GET uri = http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
141232025-04-30T06:44:17.161ZTRACtest_unauthorized (dropshot_internal): incoming request local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 87e62d1c-46ca-4da2-a429-a7881435aad1 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
141242025-04-30T06:44:17.161ZTRACtest_unauthorized (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 87e62d1c-46ca-4da2-a429-a7881435aad1 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
141252025-04-30T06:44:17.161ZDEBGtest_unauthorized (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 87e62d1c-46ca-4da2-a429-a7881435aad1 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
141262025-04-30T06:44:17.161ZDEBGtest_unauthorized (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 87e62d1c-46ca-4da2-a429-a7881435aad1 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
141272025-04-30T06:44:17.165ZINFOtest_unauthorized (dropshot_internal): request completed latency_us = 4601 local_addr = 127.0.0.1:34029 method = GET remote_addr = 127.0.0.1:40698 req_id = 87e62d1c-46ca-4da2-a429-a7881435aad1 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
141282025-04-30T06:44:17.167ZDEBGtest_unauthorized (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:34029/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "87e62d1c-46ca-4da2-a429-a7881435aad1", "content-length": "29", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
141292025-04-30T06:44:17.168ZTRACtest_unauthorized (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 8d752adb-08b9-405f-a11b-66019bf447b5
141302025-04-30T06:44:17.168ZTRACtest_unauthorized (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
141312025-04-30T06:44:17.168ZINFOtest_unauthorized (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
141322025-04-30T06:44:17.173ZDEBGtest_unauthorized (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
141332025-04-30T06:44:17.173ZDEBGtest_unauthorized (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
141342025-04-30T06:44:17.173ZDEBGtest_unauthorized (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a
141352025-04-30T06:44:17.175ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
141362025-04-30T06:44:17.176ZWARNtest_unauthorized (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
141372025-04-30T06:44:17.177ZWARNtest_unauthorized (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
141382025-04-30T06:44:17.177ZDEBGtest_unauthorized (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 505.577279ms iteration = 51 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
141392025-04-30T06:44:17.177ZDEBGtest_unauthorized (BackgroundTasks): activating background_task = inventory_collection iteration = 52 reason = Signaled
141402025-04-30T06:44:17.177ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
141412025-04-30T06:44:17.177ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
141422025-04-30T06:44:17.178ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
141432025-04-30T06:44:17.178ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
141442025-04-30T06:44:17.181ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
141452025-04-30T06:44:17.185ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
141462025-04-30T06:44:17.193ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
141472025-04-30T06:44:17.193ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
141482025-04-30T06:44:17.193ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
141492025-04-30T06:44:17.196ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
141502025-04-30T06:44:17.196ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 2 table_name = fields_string
141512025-04-30T06:44:17.196ZTRACtest_unauthorized (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
141522025-04-30T06:44:17.200ZDEBGtest_unauthorized (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
141532025-04-30T06:44:17.200ZDEBGtest_unauthorized (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
141542025-04-30T06:44:17.202ZDEBGtest_unauthorized (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1a8b2318-abf8-406d-a389-112ab91a0de8, 0)]
141552025-04-30T06:44:17.203ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
141562025-04-30T06:44:17.206ZDEBGtest_unauthorized (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 02357a36-53b1-4e9c-b024-19b15e97585a n_rows = 1 table_name = measurements_i64
141572025-04-30T06:44:17.206ZTRACtest_unauthorized (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
141582025-04-30T06:44:17.206ZTRACtest_unauthorized (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
141592025-04-30T06:44:17.207ZDEBGtest_unauthorized (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
141602025-04-30T06:44:17.207ZTRACtest_unauthorized (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
141612025-04-30T06:44:17.207ZTRACtest_unauthorized (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
141622025-04-30T06:44:17.210ZTRACtest_unauthorized (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 0d 80 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:50600 req_id = 5a58ac0d-381e-45e1-8f0d-d22436486bf6
141632025-04-30T06:44:17.210ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 3456,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:50600 req_id = 5a58ac0d-381e-45e1-8f0d-d22436486bf6
141642025-04-30T06:44:17.210ZDEBGtest_unauthorized (store): query key key = _mgs._tcp
141652025-04-30T06:44:17.210ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
141662025-04-30T06:44:17.211ZDEBGtest_unauthorized (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:50600 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 5a58ac0d-381e-45e1-8f0d-d22436486bf6
141672025-04-30T06:44:17.211ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
141682025-04-30T06:44:17.211ZDEBGtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43500, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4322, tv_nsec: 10843111 } })
141692025-04-30T06:44:17.212ZTRACtest_unauthorized (DnsResolver): lookup_service_targets: looking up SRV target
141702025-04-30T06:44:17.213ZTRACtest_unauthorized (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: b7 df 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:63204 req_id = 34a3bca3-04d5-459f-a887-af30e1999645
141712025-04-30T06:44:17.213ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 47071,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63204 req_id = 34a3bca3-04d5-459f-a887-af30e1999645
141722025-04-30T06:44:17.213ZDEBGtest_unauthorized (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
141732025-04-30T06:44:17.213ZDEBGtest_unauthorized (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:63204 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 34a3bca3-04d5-459f-a887-af30e1999645
141742025-04-30T06:44:17.287ZTRACtest_unauthorized (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
141752025-04-30T06:44:17.288ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
141762025-04-30T06:44:17.289ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
141772025-04-30T06:44:17.290ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
141782025-04-30T06:44:17.290ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
141792025-04-30T06:44:17.291ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
141802025-04-30T06:44:17.291ZWARNtest_unauthorized (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
141812025-04-30T06:44:17.292ZTRACtest_unauthorized (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 1e 32 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c .2..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:51327 req_id = 47aebd6f-3fe0-43e9-9c1f-7619d8a615bd
141822025-04-30T06:44:17.292ZDEBGtest_unauthorized (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 7730,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51327 req_id = 47aebd6f-3fe0-43e9-9c1f-7619d8a615bd
141832025-04-30T06:44:17.292ZDEBGtest_unauthorized (store): query key key = _clickhouse-admin-keeper._tcp
141842025-04-30T06:44:17.292ZERROtest_unauthorized (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 7730, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:51327 req_id = 47aebd6f-3fe0-43e9-9c1f-7619d8a615bd
141852025-04-30T06:44:17.293ZDEBGtest_unauthorized (BackgroundTasks): begin collection background_task = inventory_collection
141862025-04-30T06:44:17.293ZDEBGtest_unauthorized (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:43500
141872025-04-30T06:44:17.294ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/ignition
141882025-04-30T06:44:17.294ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
141892025-04-30T06:44:17.301ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
141902025-04-30T06:44:17.301ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
141912025-04-30T06:44:17.301ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
141922025-04-30T06:44:17.302ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
141932025-04-30T06:44:17.307ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "e6af1af0-872d-4998-a327-ed5f7fe675cf", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
141942025-04-30T06:44:17.307ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0
141952025-04-30T06:44:17.308ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
141962025-04-30T06:44:17.308ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
141972025-04-30T06:44:17.309ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
141982025-04-30T06:44:17.311ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ca8bb008-a3e8-4d26-83d9-c5ce022cd064", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
141992025-04-30T06:44:17.311ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0
142002025-04-30T06:44:17.316ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "22c22fd1-bc52-4ee3-bac9-18787e763ef1", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
142012025-04-30T06:44:17.316ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1
142022025-04-30T06:44:17.321ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0efbb724-81de-4917-a204-ab415b2e5dc5", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
142032025-04-30T06:44:17.321ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0
142042025-04-30T06:44:17.326ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9ff78cc2-30b0-489f-bf77-db1f24691120", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
142052025-04-30T06:44:17.326ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1
142062025-04-30T06:44:17.328ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "41cd29cd-39a0-4f7f-beea-fe49cb9f1591", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
142072025-04-30T06:44:17.328ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0
142082025-04-30T06:44:17.339ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8502f897-91ce-407b-884b-b82b5f48ba3b", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
142092025-04-30T06:44:17.339ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1
142102025-04-30T06:44:17.344ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f11810de-5558-4b82-9f6e-de7caebf2752", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
142112025-04-30T06:44:17.344ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cmpa
142122025-04-30T06:44:17.346ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "822efda3-626b-4111-a032-e47a304fa365", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
142132025-04-30T06:44:17.346ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
142142025-04-30T06:44:17.356ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f9de33de-4e15-4a9b-8de0-2160db8c18b9", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
142152025-04-30T06:44:17.357ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
142162025-04-30T06:44:17.358ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
142172025-04-30T06:44:17.361ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ec80775a-6d6d-4841-83c4-f25250d1e876", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
142182025-04-30T06:44:17.364ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/0/component/rot/cfpa
142192025-04-30T06:44:17.366ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d61e8bc3-aaa6-4fa5-916a-ce386097b017", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
142202025-04-30T06:44:17.366ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1
142212025-04-30T06:44:17.367ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "80775029-15b8-45e7-8fc8-4f1e447d35b8", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
142222025-04-30T06:44:17.368ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0
142232025-04-30T06:44:17.371ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c04ffc10-761e-4858-8c29-9941886a0ab9", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
142242025-04-30T06:44:17.371ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1
142252025-04-30T06:44:17.375ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "020069ff-1a27-4f4e-bb0e-23ee5fb7dd40", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
142262025-04-30T06:44:17.375ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0
142272025-04-30T06:44:17.381ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9f1211eb-94d4-46f8-9fe7-800c3beec8ab", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
142282025-04-30T06:44:17.382ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1
142292025-04-30T06:44:17.383ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "944deb07-dbac-499f-8bf7-6a480610f408", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
142302025-04-30T06:44:17.383ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0
142312025-04-30T06:44:17.387ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c1c3c6d1-42b3-4629-8378-5a8fa70c6c03", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
142322025-04-30T06:44:17.387ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1
142332025-04-30T06:44:17.390ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0a5af483-92b8-49e7-8084-02775bd2f1c7", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
142342025-04-30T06:44:17.390ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cmpa
142352025-04-30T06:44:17.391ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "13efd3b1-287f-463f-b962-bad3679598fc", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
142362025-04-30T06:44:17.391ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
142372025-04-30T06:44:17.395ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8ad58c0d-4bc8-4339-9055-6d081db6ba7f", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
142382025-04-30T06:44:17.395ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
142392025-04-30T06:44:17.396ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e6bcb334-6f2e-4029-a73a-8de302dc30a4", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
142402025-04-30T06:44:17.397ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/switch/1/component/rot/cfpa
142412025-04-30T06:44:17.398ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "48b13a57-d793-46d0-a104-ad7837ed692e", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
142422025-04-30T06:44:17.398ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0
142432025-04-30T06:44:17.400ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "905be911-0110-4af9-a19b-a232aca2acca", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
142442025-04-30T06:44:17.400ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0
142452025-04-30T06:44:17.407ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "82294f37-118c-40c7-8284-8c8b0c0f98c8", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
142462025-04-30T06:44:17.407ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1
142472025-04-30T06:44:17.410ZTRACtest_unauthorized (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
142482025-04-30T06:44:17.410ZTRACtest_unauthorized (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
142492025-04-30T06:44:17.410ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
142502025-04-30T06:44:17.410ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
142512025-04-30T06:44:17.412ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
142522025-04-30T06:44:17.413ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a8064bb8-dfaa-4feb-8b1d-efad7a616c61", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
142532025-04-30T06:44:17.413ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0
142542025-04-30T06:44:17.416ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
142552025-04-30T06:44:17.429ZDEBGtest_unauthorized (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
142562025-04-30T06:44:17.429ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
142572025-04-30T06:44:17.429ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
142582025-04-30T06:44:17.432ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
142592025-04-30T06:44:17.434ZTRACtest_unauthorized (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
142602025-04-30T06:44:17.435ZDEBGtest_unauthorized (InternalApi): roles roles = RoleSet { roles: {} }
142612025-04-30T06:44:17.435ZDEBGtest_unauthorized (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
142622025-04-30T06:44:17.437ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3b28cfd3-e4b6-4c38-8b1a-86af8db13d94", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
142632025-04-30T06:44:17.437ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1
142642025-04-30T06:44:17.440ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "efcf409b-e302-4f73-9643-7b089295a32b", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
142652025-04-30T06:44:17.441ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0
142662025-04-30T06:44:17.444ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "525456e9-7b07-48ac-af49-84309a6a8b7b", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
142672025-04-30T06:44:17.444ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1
142682025-04-30T06:44:17.446ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "65072498-0256-4ff2-9e10-a63c08c609cf", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
142692025-04-30T06:44:17.446ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cmpa
142702025-04-30T06:44:17.449ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b61e3f46-e5ff-42d1-94c9-1e16dcda4d13", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
142712025-04-30T06:44:17.449ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
142722025-04-30T06:44:17.449ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "874699dd-695b-49cf-85bd-3e3261ed99a7", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
142732025-04-30T06:44:17.449ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
142742025-04-30T06:44:17.452ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5087247e-35a4-4fd9-a7c2-49cd4fd00cbd", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
142752025-04-30T06:44:17.452ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/0/component/rot/cfpa
142762025-04-30T06:44:17.453ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "938c2e9e-0664-4eba-a393-18ca803b3acc", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
142772025-04-30T06:44:17.453ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1
142782025-04-30T06:44:17.454ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7cdb8d3a-02b1-4bfa-acad-00d990eaf8b5", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
142792025-04-30T06:44:17.455ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0
142802025-04-30T06:44:17.458ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3b4c5cd4-b72b-4414-b680-a0bfee1d2447", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
142812025-04-30T06:44:17.458ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1
142822025-04-30T06:44:17.460ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7661e633-bc5e-47bc-928d-ea54562c9ec5", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
142832025-04-30T06:44:17.460ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0
142842025-04-30T06:44:17.462ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f22e37bc-5544-45e9-9b38-327cc1aa8509", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
142852025-04-30T06:44:17.462ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1
142862025-04-30T06:44:17.465ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "09a88360-f475-492d-bb3a-44d774d7adf5", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
142872025-04-30T06:44:17.465ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0
142882025-04-30T06:44:17.469ZDEBGtest_unauthorized: attempt to fetch latest inventory collection result = Ok("found")
142892025-04-30T06:44:17.472ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "76170254-9153-4f75-9ac1-a2a7336fc7b2", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
142902025-04-30T06:44:17.472ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1
142912025-04-30T06:44:17.476ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9c3218ab-7a6c-4bc5-b8cf-0ffcfd4814df", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
142922025-04-30T06:44:17.476ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cmpa
142932025-04-30T06:44:17.477ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f529a0c9-e768-4a0c-9b9a-e40ca51c53da", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
142942025-04-30T06:44:17.478ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
142952025-04-30T06:44:17.479ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "bcae8350-54dd-4c29-902c-8f28fbf71c93", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
142962025-04-30T06:44:17.479ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
142972025-04-30T06:44:17.480ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a7b360ab-5af0-436d-8367-4c85a91b8c64", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
142982025-04-30T06:44:17.480ZDEBGtest_unauthorized (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:43500 method = GET uri = http://[::1]:43500/sp/sled/1/component/rot/cfpa
142992025-04-30T06:44:17.483ZDEBGtest_unauthorized (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:43500 result = Ok(Response { url: "http://[::1]:43500/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6e625452-be5a-4734-aa49-53d7a78d9a1e", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
143002025-04-30T06:44:17.483ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
143012025-04-30T06:44:17.483ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
143022025-04-30T06:44:17.483ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
143032025-04-30T06:44:17.483ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
143042025-04-30T06:44:17.485ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
143052025-04-30T06:44:17.488ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143062025-04-30T06:44:17.495ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
143072025-04-30T06:44:17.495ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
143082025-04-30T06:44:17.495ZTRACtest_unauthorized (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
143092025-04-30T06:44:17.495ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
143102025-04-30T06:44:17.495ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
143112025-04-30T06:44:17.496ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
143122025-04-30T06:44:17.498ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143132025-04-30T06:44:17.503ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
143142025-04-30T06:44:17.503ZTRACtest_unauthorized (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
143152025-04-30T06:44:17.503ZDEBGtest_unauthorized (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
143162025-04-30T06:44:17.503ZDEBGtest_unauthorized (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
143172025-04-30T06:44:17.569ZDEBGtest_unauthorized (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:57786
143182025-04-30T06:44:17.570ZDEBGtest_unauthorized (BackgroundTasks): client request SledAgent = http://[::1]:57786 background_task = inventory_collection body = None method = GET uri = http://[::1]:57786/inventory