Max level shown:
12025-04-30T06:13:57.429ZDEBGtest_basic (init_with_steps): Running step start_crdb
22025-04-30T06:13:57.430ZDEBGtest_basic: Starting CRDB
32025-04-30T06:13:57.453ZINFOtest_basic: cockroach temporary directory: /var/tmp/omicron_tmp/.tmp5G0CxG
42025-04-30T06:13:57.453ZINFOtest_basic: cockroach: copying from seed tarball (/var/tmp/omicron_tmp/crdb-base-build/f234518892194a05ad8767dc97f01a61a55baf1431942a314ce8b0f72b18ec2e.tar) to storage directory (/var/tmp/omicron_tmp/.tmp5G0CxG/data)
52025-04-30T06:13:57.471ZINFOtest_basic: cockroach command line: cockroach start-single-node --insecure --http-addr=:0 --store=path=/var/tmp/omicron_tmp/.tmp5G0CxG/data,ballast-size=0 --listen-addr [::1]:0 --listening-url-file /var/tmp/omicron_tmp/.tmp5G0CxG/listen-url
62025-04-30T06:13:57.471ZINFOtest_basic: cockroach environment: BUILDOMAT_JOB_ID=01JT2JWKGHF3JHFMW0JR13NSA6 BUILDOMAT_TASK_ID=4 CARGO=/home/build/.rustup/toolchains/1.86.0-x86_64-unknown-illumos/bin/cargo CARGO_HOME=/home/build/.cargo CARGO_INCREMENTAL=0 CARGO_MANIFEST_DIR=/work/oxidecomputer/omicron/nexus CARGO_PKG_AUTHORS= CARGO_PKG_DESCRIPTION= CARGO_PKG_HOMEPAGE= CARGO_PKG_LICENSE=MPL-2.0 CARGO_PKG_LICENSE_FILE= CARGO_PKG_NAME=omicron-nexus CARGO_PKG_REPOSITORY= CARGO_PKG_RUST_VERSION= CARGO_PKG_VERSION=0.1.0 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=1 CARGO_PKG_VERSION_PATCH=0 CARGO_PKG_VERSION_PRE= CARGO_TERM_COLOR=always CI=true CRDB_SEED_TAR=/var/tmp/omicron_tmp/crdb-base-build/f234518892194a05ad8767dc97f01a61a55baf1431942a314ce8b0f72b18ec2e.tar GITHUB_REPOSITORY=oxidecomputer/omicron GITHUB_SHA=531912b3fc900900bc7bd18f5eac1ce1e22cb1cd GOTRACEBACK=crash HOME=/home/build LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8 LD_LIBRARY_PATH=/work/oxidecomputer/omicron/target/debug/build/aws-lc-sys-6532f52db341a7cc/out/build/artifacts:/work/oxidecomputer/omicron/target/debug/build/blake3-403ee89fc24db8f8/out:/work/oxidecomputer/omicron/target/debug/build/bzip2-sys-aece66bfb5c8df92/out/lib:/work/oxidecomputer/omicron/target/debug/build/libgit2-sys-5fe90ecb72e1690f/out/build:/work/oxidecomputer/omicron/target/debug/build/ring-fb368fe91b2bd145/out:/work/oxidecomputer/omicron/target/debug/build/tofino-ffe28f7f2301d0c2/out:/work/oxidecomputer/omicron/target/debug/build/zstd-sys-a7fb19a1e17b70f6/out:/work/oxidecomputer/omicron/target/debug/deps:/work/oxidecomputer/omicron/target/debug:/home/build/.rustup/toolchains/1.86.0-x86_64-unknown-illumos/lib/rustlib/x86_64-unknown-illumos/lib:/home/build/.rustup/toolchains/1.86.0-x86_64-unknown-illumos/lib LOGNAME=build NEXTEST=1 NEXTEST_EXECUTION_MODE=process-per-test NEXTEST_LD_LIBRARY_PATH=/work/oxidecomputer/omicron/target/debug/build/aws-lc-sys-6532f52db341a7cc/out/build/artifacts:/work/oxidecomputer/omicron/target/debug/build/blake3-403ee89fc24db8f8/out:/work/oxidecomputer/omicron/target/debug/build/bzip2-sys-aece66bfb5c8df92/out/lib:/work/oxidecomputer/omicron/target/debug/build/libgit2-sys-5fe90ecb72e1690f/out/build:/work/oxidecomputer/omicron/target/debug/build/ring-fb368fe91b2bd145/out:/work/oxidecomputer/omicron/target/debug/build/tofino-ffe28f7f2301d0c2/out:/work/oxidecomputer/omicron/target/debug/build/zstd-sys-a7fb19a1e17b70f6/out:/work/oxidecomputer/omicron/target/debug/deps:/work/oxidecomputer/omicron/target/debug:/home/build/.rustup/toolchains/1.86.0-x86_64-unknown-illumos/lib/rustlib/x86_64-unknown-illumos/lib:/home/build/.rustup/toolchains/1.86.0-x86_64-unknown-illumos/lib NEXTEST_PROFILE=ci NEXTEST_RUN_ID=29735cc2-0883-4059-8ca2-1da53f01ede3 NEXTEST_TEST_GLOBAL_SLOT=0 NEXTEST_TEST_GROUP=@global NEXTEST_TEST_GROUP_SLOT=none OUT_DIR=/work/oxidecomputer/omicron/target/debug/build/omicron-nexus-553062f859dbbfa0/out PATH=/work/oxidecomputer/omicron/out/mgd/root/opt/oxide/mgd/bin:/work/oxidecomputer/omicron/out/dendrite-stub/bin:/work/oxidecomputer/omicron/out/clickhouse:/work/oxidecomputer/omicron/out/cockroachdb/bin:/home/build/.cargo/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/ooce/bin:/opt/ooce/sbin PWD=/work/oxidecomputer/omicron RUSTC_BOOTSTRAP=1 RUSTDOCFLAGS=--document-private-items -D warnings RUSTFLAGS=-D warnings -C link-arg=-R/usr/platform/oxide/lib/amd64 RUSTUP_HOME=/home/build/.rustup RUSTUP_TOOLCHAIN=1.86.0-x86_64-unknown-illumos RUST_BACKTRACE=1 RUST_RECURSION_COUNT=1 SHLVL=1 SSL_CERT_DIR=/usr/ssl/certs SSL_CERT_FILE=/etc/ssl/cacert.pem TMPDIR=/var/tmp/omicron_tmp TZ=UTC USER=build _=/usr/bin/ptime __NEXTEST_ATTEMPT=1
72025-04-30T06:13:59.247ZINFOtest_basic: cockroach pid: 18325
82025-04-30T06:13:59.248ZINFOtest_basic: cockroach listen URL: postgresql://root@[::1]:47002/omicron?sslmode=disable
92025-04-30T06:13:59.248ZDEBGtest_basic (init_with_steps): Running step start_clickhouse
102025-04-30T06:13:59.248ZDEBGtest_basic: Starting Clickhouse
112025-04-30T06:13:59.998ZDEBGtest_basic (init_with_steps): Running step start_gateway_switch0
122025-04-30T06:13:59.998ZDEBGtest_basic: Starting Management Gateway
132025-04-30T06:14:00.122ZDEBGtest_basic (init_with_steps): Running step start_dendrite_switch0
142025-04-30T06:14:00.122ZDEBGtest_basic: Starting Dendrite for switch0
152025-04-30T06:14:00.177ZDEBGtest_basic (init_with_steps): Running step start_mgd_switch0
162025-04-30T06:14:00.177ZDEBGtest_basic: Starting mgd for switch0
172025-04-30T06:14:00.545ZDEBGtest_basic: mgd port is 51711
182025-04-30T06:14:00.546ZDEBGtest_basic (init_with_steps): Running step record_switch_dns
192025-04-30T06:14:00.547ZDEBGtest_basic: Recording DNS for the switch zones sled_id = b6d65341-167c-41df-9b5c-41cded99c229 switch_location = switch0
202025-04-30T06:14:00.547ZDEBGtest_basic (init_with_steps): Running step start_internal_dns
212025-04-30T06:14:00.547ZINFOtest_basic (store): opening sled database path = /var/tmp/omicron_tmp/.tmppyKRzQ
222025-04-30T06:14:00.723ZINFOtest_basic (store): pruning trees for generations newer than 0
232025-04-30T06:14:00.723ZINFOtest_basic (store): pruning trees for generations older than 0 keep = 3
242025-04-30T06:14:00.723ZINFOtest_basic (dns): DNS server bound to address kind = dns local_address = [::1]:64728
252025-04-30T06:14:00.725ZDEBGtest_basic (http): registered endpoint kind = dns local_addr = [::1]:57818 method = GET path = /config versions = all
262025-04-30T06:14:00.725ZDEBGtest_basic (http): registered endpoint kind = dns local_addr = [::1]:57818 method = PUT path = /config versions = all
272025-04-30T06:14:00.725ZINFOtest_basic (http): listening kind = dns local_addr = [::1]:57818
282025-04-30T06:14:00.727ZDEBGtest_basic (http): successfully registered DTrace USDT probes kind = dns local_addr = [::1]:57818
292025-04-30T06:14:00.727ZDEBGtest_basic (init_with_steps): Running step start_external_dns
302025-04-30T06:14:00.727ZINFOtest_basic (store): opening sled database path = /var/tmp/omicron_tmp/.tmpei5xx4
312025-04-30T06:14:00.748ZINFOtest_basic (store): pruning trees for generations newer than 0
322025-04-30T06:14:00.748ZINFOtest_basic (store): pruning trees for generations older than 0 keep = 3
332025-04-30T06:14:00.748ZINFOtest_basic (dns): DNS server bound to address kind = dns local_address = [::1]:56755
342025-04-30T06:14:00.749ZDEBGtest_basic (http): registered endpoint kind = dns local_addr = [::1]:46146 method = GET path = /config versions = all
352025-04-30T06:14:00.749ZDEBGtest_basic (http): registered endpoint kind = dns local_addr = [::1]:46146 method = PUT path = /config versions = all
362025-04-30T06:14:00.749ZINFOtest_basic (http): listening kind = dns local_addr = [::1]:46146
372025-04-30T06:14:00.750ZDEBGtest_basic (http): successfully registered DTrace USDT probes kind = dns local_addr = [::1]:46146
382025-04-30T06:14:00.750ZDEBGtest_basic (init_with_steps): Running step start_nexus_internal
392025-04-30T06:14:00.750ZDEBGtest_basic: Starting Nexus (internal API)
402025-04-30T06:14:00.751ZINFOtest_basic: setting up nexus server
412025-04-30T06:14:00.753ZINFOtest_basic (ServerContext): registering Oso class class = Action
422025-04-30T06:14:00.753ZINFOtest_basic (ServerContext): registering Oso class class = AnyActor
432025-04-30T06:14:00.753ZINFOtest_basic (ServerContext): registering Oso class class = AuthenticatedActor
442025-04-30T06:14:00.753ZINFOtest_basic (ServerContext): registering Oso class class = BlueprintConfig
452025-04-30T06:14:00.753ZINFOtest_basic (ServerContext): registering Oso class class = Database
462025-04-30T06:14:00.753ZINFOtest_basic (ServerContext): registering Oso class class = DnsConfig
472025-04-30T06:14:00.753ZINFOtest_basic (ServerContext): registering Oso class class = Fleet
482025-04-30T06:14:00.753ZINFOtest_basic (ServerContext): registering Oso class class = Inventory
492025-04-30T06:14:00.754ZINFOtest_basic (ServerContext): registering Oso class class = IpPoolList
502025-04-30T06:14:00.754ZINFOtest_basic (ServerContext): registering Oso class class = ConsoleSessionList
512025-04-30T06:14:00.754ZINFOtest_basic (ServerContext): registering Oso class class = DeviceAuthRequestList
522025-04-30T06:14:00.754ZINFOtest_basic (ServerContext): registering Oso class class = SiloCertificateList
532025-04-30T06:14:00.754ZINFOtest_basic (ServerContext): registering Oso class class = SiloIdentityProviderList
542025-04-30T06:14:00.754ZINFOtest_basic (ServerContext): registering Oso class class = SiloUserList
552025-04-30T06:14:00.754ZINFOtest_basic (ServerContext): registering Oso class class = TargetReleaseConfig
562025-04-30T06:14:00.754ZINFOtest_basic (ServerContext): registering Oso class class = WebhookEventClassList
572025-04-30T06:14:00.755ZINFOtest_basic (ServerContext): registering Oso class class = Project
582025-04-30T06:14:00.755ZINFOtest_basic (ServerContext): registering Oso class class = Disk
592025-04-30T06:14:00.755ZINFOtest_basic (ServerContext): registering Oso class class = Snapshot
602025-04-30T06:14:00.755ZINFOtest_basic (ServerContext): registering Oso class class = ProjectImage
612025-04-30T06:14:00.755ZINFOtest_basic (ServerContext): registering Oso class class = AffinityGroup
622025-04-30T06:14:00.756ZINFOtest_basic (ServerContext): registering Oso class class = AntiAffinityGroup
632025-04-30T06:14:00.756ZINFOtest_basic (ServerContext): registering Oso class class = Instance
642025-04-30T06:14:00.756ZINFOtest_basic (ServerContext): registering Oso class class = IpPool
652025-04-30T06:14:00.756ZINFOtest_basic (ServerContext): registering Oso class class = InstanceNetworkInterface
662025-04-30T06:14:00.756ZINFOtest_basic (ServerContext): registering Oso class class = Vpc
672025-04-30T06:14:00.756ZINFOtest_basic (ServerContext): registering Oso class class = VpcRouter
682025-04-30T06:14:00.756ZINFOtest_basic (ServerContext): registering Oso class class = InternetGateway
692025-04-30T06:14:00.756ZINFOtest_basic (ServerContext): registering Oso class class = InternetGatewayIpPool
702025-04-30T06:14:00.756ZINFOtest_basic (ServerContext): registering Oso class class = InternetGatewayIpAddress
712025-04-30T06:14:00.756ZINFOtest_basic (ServerContext): registering Oso class class = RouterRoute
722025-04-30T06:14:00.756ZINFOtest_basic (ServerContext): registering Oso class class = VpcSubnet
732025-04-30T06:14:00.756ZINFOtest_basic (ServerContext): registering Oso class class = FloatingIp
742025-04-30T06:14:00.756ZINFOtest_basic (ServerContext): registering Oso class class = Image
752025-04-30T06:14:00.756ZINFOtest_basic (ServerContext): registering Oso class class = SiloImage
762025-04-30T06:14:00.756ZINFOtest_basic (ServerContext): registering Oso class class = AddressLot
772025-04-30T06:14:00.756ZINFOtest_basic (ServerContext): registering Oso class class = Blueprint
782025-04-30T06:14:00.756ZINFOtest_basic (ServerContext): registering Oso class class = LoopbackAddress
792025-04-30T06:14:00.756ZINFOtest_basic (ServerContext): registering Oso class class = Certificate
802025-04-30T06:14:00.756ZINFOtest_basic (ServerContext): registering Oso class class = ConsoleSession
812025-04-30T06:14:00.756ZINFOtest_basic (ServerContext): registering Oso class class = DeviceAuthRequest
822025-04-30T06:14:00.756ZINFOtest_basic (ServerContext): registering Oso class class = DeviceAccessToken
832025-04-30T06:14:00.756ZINFOtest_basic (ServerContext): registering Oso class class = PhysicalDisk
842025-04-30T06:14:00.756ZINFOtest_basic (ServerContext): registering Oso class class = Rack
852025-04-30T06:14:00.756ZINFOtest_basic (ServerContext): registering Oso class class = RoleBuiltin
862025-04-30T06:14:00.756ZINFOtest_basic (ServerContext): registering Oso class class = SshKey
872025-04-30T06:14:00.756ZINFOtest_basic (ServerContext): registering Oso class class = Silo
882025-04-30T06:14:00.756ZINFOtest_basic (ServerContext): registering Oso class class = SiloUser
892025-04-30T06:14:00.756ZINFOtest_basic (ServerContext): registering Oso class class = SiloGroup
902025-04-30T06:14:00.756ZINFOtest_basic (ServerContext): registering Oso class class = SupportBundle
912025-04-30T06:14:00.756ZINFOtest_basic (ServerContext): registering Oso class class = IdentityProvider
922025-04-30T06:14:00.756ZINFOtest_basic (ServerContext): registering Oso class class = SamlIdentityProvider
932025-04-30T06:14:00.756ZINFOtest_basic (ServerContext): registering Oso class class = Sled
942025-04-30T06:14:00.756ZINFOtest_basic (ServerContext): registering Oso class class = TufRepo
952025-04-30T06:14:00.756ZINFOtest_basic (ServerContext): registering Oso class class = TufArtifact
962025-04-30T06:14:00.756ZINFOtest_basic (ServerContext): registering Oso class class = WebhookEvent
972025-04-30T06:14:00.756ZINFOtest_basic (ServerContext): registering Oso class class = WebhookReceiver
982025-04-30T06:14:00.756ZINFOtest_basic (ServerContext): registering Oso class class = WebhookSecret
992025-04-30T06:14:00.756ZINFOtest_basic (ServerContext): registering Oso class class = Zpool
1002025-04-30T06:14:00.756ZINFOtest_basic (ServerContext): registering Oso class class = Service
1012025-04-30T06:14:00.756ZINFOtest_basic (ServerContext): registering Oso class class = UserBuiltin
1022025-04-30T06:14:00.757ZINFOtest_basic (ServerContext): full Oso configuration config = #\n# Oso configuration for Omicron\n# This file is augmented by generated snippets.\n#\n\n#\n# ACTOR TYPES AND BASIC RULES\n#\n\n# `AnyActor` includes both authenticated and unauthenticated users.\nactor AnyActor {}\n\n# An `AuthenticatedActor` has an identity in the system. All of our operations\n# today require that an actor be authenticated.\nactor AuthenticatedActor {}\n\n# For any resource, `actor` can perform action `action` on it if they're\n# authenticated and their role(s) give them the corresponding permission on that\n# resource.\nallow(actor: AnyActor, action: Action, resource) if\n actor.authenticated and\n has_permission(actor.authn_actor.unwrap(), action.to_perm(), resource);\n\n# Define role relationships\nhas_role(actor: AuthenticatedActor, role: String, resource: Resource)\n\tif resource.has_role(actor, role);\n\n#\n# ROLES AND PERMISSIONS IN THE FLEET/SILO/PROJECT HIERARCHY\n#\n# We define the following permissions for most resources in the system:\n#\n# - "create_child": required to create child resources (of any type)\n#\n# - "list_children": required to list child resources (of all types) of a\n# resource\n#\n# - "modify": required to modify or delete a resource\n#\n# - "read": required to read a resource\n#\n# We define the following predefined roles for only a few high-level resources:\n# the Fleet (see below), Silo, Organization, and Project. The specific roles\n# are oriented around intended use-cases:\n#\n# - "admin": has all permissions on the resource\n#\n# - "collaborator": has "read", "list_children", and "create_child", plus\n# the "admin" role for child resources. The idea is that if you're an\n# Organization Collaborator, you have full control over the Projects within\n# the Organization, but you cannot modify or delete the Organization itself.\n#\n# - "viewer": has "read" and "list_children" on a resource\n#\n# Below the Project level, permissions are granted via roles at the Project\n# level. For example, for someone to be able to create, modify, or delete any\n# Instances, they must be granted project.collaborator, which means they can\n# create, modify, or delete _all_ resources in the Project.\n#\n# The complete set of predefined roles:\n#\n# - fleet.admin (superuser for the whole system)\n# - fleet.collaborator (can manage Silos)\n# - fleet.viewer (can read most non-siloed resources in the system)\n# - silo.admin (superuser for the silo)\n# - silo.collaborator (can create and own Organizations)\n# - silo.viewer (can read most resources within the Silo)\n# - organization.admin (complete control over an organization)\n# - organization.collaborator (can manage Projects)\n# - organization.viewer (can read most resources within the Organization)\n# - project.admin (complete control over a Project)\n# - project.collaborator (can manage all resources within the Project)\n# - project.viewer (can read most resources within the Project)\n#\n# Outside the Silo/Organization/Project hierarchy, we (currently) treat most\n# resources as nested under Fleet or else a synthetic resource (see below). We\n# do not yet support role assignments on anything other than Fleet, Silo,\n# Organization, or Project.\n#\n\n# "Fleet" is a global singleton representing the whole system. The name comes\n# from the idea described in RFD 24, but it's not quite right. This probably\n# should be more like "Region" or "AvailabilityZone". The precise boundaries\n# have not yet been figured out.\nresource Fleet {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\troles = [\n\t # Roles that can be attached by users\n\t "admin",\n\t "collaborator",\n\t "viewer",\n\n\t # Internal-only roles\n\t "external-authenticator"\n\t];\n\n\t# Roles implied by other roles on this resource\n\t"viewer" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\t"create_child" if "collaborator";\n\t"modify" if "admin";\n}\n\n# For fleets specifically, roles can be conferred by roles on the user's Silo.\nhas_role(actor: AuthenticatedActor, role: String, _: Fleet) if\n\tsilo_role in actor.confers_fleet_role(role) and\n\thas_role(actor, silo_role, actor.silo.unwrap());\n\nresource Silo {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\troles = [ "admin", "collaborator", "viewer" ];\n\n\t# Roles implied by other roles on this resource\n\t"viewer" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\n\t"create_child" if "collaborator";\n\t"modify" if "admin";\n\n\t# Permissions implied by roles on this resource's parent (Fleet). Fleet\n\t# privileges allow a user to see and potentially administer the Silo,\n\t# but they do not give anyone permission to look at anything inside the\n\t# Silo. To achieve this, we use permission rules here. (If we granted\n\t# Fleet administrators _roles_ on the Silo, then those would cascade\n\t# into the Silo as well.)\n\trelations = { parent_fleet: Fleet };\n\t"read" if "viewer" on "parent_fleet";\n\t"modify" if "collaborator" on "parent_fleet";\n\n\t# external authenticator has to create silo users\n\t"list_children" if "external-authenticator" on "parent_fleet";\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\n\nhas_relation(fleet: Fleet, "parent_fleet", silo: Silo)\n\tif silo.fleet = fleet;\n\n# As a special case, all authenticated users can read their own Silo. That's\n# not quite the same as having the "viewer" role. For example, they cannot list\n# Organizations in the Silo.\n#\n# One reason this is necessary is because if an unprivileged user tries to\n# create an Organization using "POST /organizations", they should get back a 403\n# (which implies they're able to see /organizations, which is essentially seeing\n# the Silo itself) rather than a 404. This behavior isn't a hard constraint\n# (i.e., you could reasonably get a 404 for an API you're not allowed to call).\n# Nor is the implementation (i.e., we could special-case this endpoint somehow).\n# But granting this permission is the simplest way to keep this endpoint's\n# behavior consistent with the rest of the API.\n#\n# This rule is also used to determine if a user can list the identity providers\n# in the Silo (which they should be able to), since that's predicated on being\n# able to read the Silo.\n#\n# It's unclear what else would break if users couldn't see their own Silo.\nhas_permission(actor: AuthenticatedActor, "read", silo: Silo)\n\tif silo in actor.silo;\n\nresource Project {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\troles = [ "admin", "collaborator", "viewer" ];\n\n\t# Roles implied by other roles on this resource\n\t"viewer" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\t"create_child" if "collaborator";\n\t"modify" if "admin";\n\n\t# Roles implied by roles on this resource's parent (Silo)\n\trelations = { parent_silo: Silo };\n\t"admin" if "collaborator" on "parent_silo";\n\t"viewer" if "viewer" on "parent_silo";\n}\nhas_relation(silo: Silo, "parent_silo", project: Project)\n\tif project.silo = silo;\n\n#\n# GENERAL RESOURCES OUTSIDE THE SILO/PROJECT HIERARCHY\n#\n# Many resources use snippets of Polar generated by the `authz_resource!` Rust\n# macro. Some resources require custom Polar code. Those appear here.\n#\n\nresource Certificate {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Fleet-level and silo-level roles both grant privileges on certificates.\n\t"read" if "admin" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"read" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", certificate: Certificate)\n\tif certificate.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", certificate: Certificate)\n\tif certificate.silo.fleet = fleet;\n\nresource SiloUser {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\t# Fleet and Silo administrators can manage a Silo's users. This is one\n\t# of the only areas of Silo configuration that Fleet Administrators have\n\t# permissions on.\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\t"list_children" if "read" on "parent_silo";\n\t"read" if "read" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\t"list_children" if "read" on "parent_fleet";\n\t"read" if "read" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", user: SiloUser)\n\tif user.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", user: SiloUser)\n\tif user.silo.fleet = fleet;\n\n# authenticated actors have all permissions on themselves\nhas_permission(actor: AuthenticatedActor, _perm: String, silo_user: SiloUser)\n if actor.equals_silo_user(silo_user);\n\nhas_permission(actor: AuthenticatedActor, "read", silo_user: SiloUser)\n if silo_user.silo in actor.silo;\n\nresource SiloGroup {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\trelations = { parent_silo: Silo };\n\t"list_children" if "read" on "parent_silo";\n\t"read" if "read" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n}\nhas_relation(silo: Silo, "parent_silo", group: SiloGroup)\n\tif group.silo = silo;\n\nresource SshKey {\n\tpermissions = [ "read", "modify" ];\n\trelations = { silo_user: SiloUser };\n\n\t"read" if "read" on "silo_user";\n\t"modify" if "modify" on "silo_user";\n}\nhas_relation(user: SiloUser, "silo_user", ssh_key: SshKey)\n\tif ssh_key.silo_user = user;\n\nresource IdentityProvider {\n\tpermissions = [\n\t "read",\n\t "modify",\n\t "create_child",\n\t "list_children",\n\t];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Silo-level roles grant privileges on identity providers.\n\t"read" if "viewer" on "parent_silo";\n\t"list_children" if "viewer" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\n\t# Fleet-level roles also grant privileges on identity providers.\n\t"read" if "viewer" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", identity_provider: IdentityProvider)\n\tif identity_provider.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: IdentityProvider)\n\tif collection.silo.fleet = fleet;\n\nresource SamlIdentityProvider {\n\tpermissions = [\n\t "read",\n\t "modify",\n\t "create_child",\n\t "list_children",\n\t];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Silo-level roles grant privileges on identity providers.\n\t"read" if "viewer" on "parent_silo";\n\t"list_children" if "viewer" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\n\t# Fleet-level roles also grant privileges on identity providers.\n\t"read" if "viewer" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", saml_identity_provider: SamlIdentityProvider)\n\tif saml_identity_provider.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SamlIdentityProvider)\n\tif collection.silo.fleet = fleet;\n\n#\n# SYNTHETIC RESOURCES OUTSIDE THE SILO HIERARCHY\n#\n# The resources here do not correspond to anything that appears explicitly in\n# the API or is stored in the database. These are used either at the top level\n# of the API path (e.g., "/v1/system/ip-pools") or as an implementation detail of the system\n# (in the case of console sessions and "Database"). The policies are\n# either statically-defined in this file or driven by role assignments on the\n# Fleet. None of these resources defines their own roles.\n#\n\n# Describes the policy for reading and modifying DNS configuration\n# (both internal and external)\nresource DnsConfig {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_fleet: Fleet };\n\t# "external-authenticator" requires these permissions because that's the\n\t# context that Nexus uses when creating and deleting Silos. These\n\t# operations necessarily need to read and modify DNS configuration.\n\t"read" if "external-authenticator" on "parent_fleet";\n\t"modify" if "external-authenticator" on "parent_fleet";\n\t# "admin" on the parent fleet also gets these permissions, primarily for\n\t# the test suite.\n\t"read" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", dns_config: DnsConfig)\n\tif dns_config.fleet = fleet;\n\n# Describes the policy for accessing blueprints\nresource BlueprintConfig {\n\tpermissions = [\n\t "list_children", # list blueprints\n\t "create_child", # create blueprint\n\t "read", # read the current target\n\t "modify", # change the current target\n\t];\n\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"read" if "viewer" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", list: BlueprintConfig)\n\tif list.fleet = fleet;\n\n# Describes the policy for accessing blueprints\nresource TargetReleaseConfig {\n\tpermissions = [\n\t "read", # read the current target release\n\t "modify", # change the current target release\n\t];\n\n\trelations = { parent_fleet: Fleet };\n\t"read" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", resource: TargetReleaseConfig)\n\tif resource.fleet = fleet;\n\n# Describes the policy for reading and modifying low-level inventory\nresource Inventory {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_fleet: Fleet };\n\t"read" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", inventory: Inventory)\n\tif inventory.fleet = fleet;\n\n# Describes the policy for accessing "/v1/system/ip-pools" in the API\nresource IpPoolList {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "create_child",\n\t];\n\n\t# Fleet Administrators can create or modify the IP Pools list.\n\trelations = { parent_fleet: Fleet };\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n\n\t# Fleet Viewers can list IP Pools\n\t"list_children" if "viewer" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", ip_pool_list: IpPoolList)\n\tif ip_pool_list.fleet = fleet;\n\n# Any authenticated user can create a child of a provided IP Pool.\n# This is necessary to use the pools when provisioning instances.\nhas_permission(actor: AuthenticatedActor, "create_child", ip_pool: IpPool)\n\tif silo in actor.silo and silo.fleet = ip_pool.fleet;\n\n# Describes the policy for creating and managing web console sessions.\nresource ConsoleSessionList {\n\tpermissions = [ "create_child" ];\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", collection: ConsoleSessionList)\n\tif collection.fleet = fleet;\n\n# Describes the policy for creating and managing device authorization requests.\nresource DeviceAuthRequestList {\n\tpermissions = [ "create_child" ];\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", collection: DeviceAuthRequestList)\n\tif collection.fleet = fleet;\n\n# Describes the policy for creating and managing Silo certificates\nresource SiloCertificateList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Both Fleet and Silo administrators can see and modify the Silo's\n\t# certificates.\n\t"list_children" if "admin" on "parent_silo";\n\t"list_children" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloCertificateList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloCertificateList)\n\tif collection.silo.fleet = fleet;\n\n# Describes the policy for creating and managing Silo identity providers\nresource SiloIdentityProviderList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Everyone who can read the Silo (which includes all the users in the\n\t# Silo) can see the identity providers in it.\n\t"list_children" if "read" on "parent_silo";\n\n\t# Fleet and Silo administrators can manage the Silo's identity provider\n\t# configuration. This is one of the only areas of Silo configuration\n\t# that Fleet Administrators have permissions on. This is also one of\n\t# the only cases where we need to look two levels up the hierarchy to\n\t# see if somebody has the right permission. For most other things,\n\t# permissions cascade down the hierarchy so we only need to look at the\n\t# parent.\n\t"create_child" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloIdentityProviderList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloIdentityProviderList)\n\tif collection.silo.fleet = fleet;\n\n# Describes the policy for creating and managing Silo users (mostly intended for\n# API-managed users)\nresource SiloUserList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Everyone who can read the Silo (which includes all the users in the\n\t# Silo) can see the users in it.\n\t"list_children" if "read" on "parent_silo";\n\n\t# Fleet and Silo administrators can manage the Silo's users. This is\n\t# one of the only areas of Silo configuration that Fleet Administrators\n\t# have permissions on. This is also one of the few cases (so far) where\n\t# we need to look two levels up the hierarchy to see if somebody has the\n\t# right permission. For most other things, permissions cascade down the\n\t# hierarchy so we only need to look at the parent.\n\t"create_child" if "admin" on "parent_silo";\n\t"list_children" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloUserList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloUserList)\n\tif collection.silo.fleet = fleet;\n\n# These rules grants the external authenticator role the permissions it needs to\n# read silo users and modify their sessions. This is necessary for login to\n# work.\nhas_permission(actor: AuthenticatedActor, "read", silo: Silo)\n\tif has_role(actor, "external-authenticator", silo.fleet);\nhas_permission(actor: AuthenticatedActor, "read", user: SiloUser)\n\tif has_role(actor, "external-authenticator", user.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", user: SiloUser)\n\tif has_role(actor, "external-authenticator", user.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "read", group: SiloGroup)\n\tif has_role(actor, "external-authenticator", group.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", group: SiloGroup)\n\tif has_role(actor, "external-authenticator", group.silo.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", session: ConsoleSession)\n\tif has_role(actor, "external-authenticator", session.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", session: ConsoleSession)\n\tif has_role(actor, "external-authenticator", session.fleet);\n\n# All authenticated users can read and delete device authn requests because\n# by necessity these operations happen before we've figured out what user (or\n# even Silo) the device auth is associated with. Any user can claim a device\n# auth request with the right user code (that's how it works) -- it's the user\n# code and associated logic that prevents unauthorized access here.\nhas_permission(_actor: AuthenticatedActor, "read", _device_auth: DeviceAuthRequest);\nhas_permission(_actor: AuthenticatedActor, "modify", _device_auth: DeviceAuthRequest);\n\nhas_permission(actor: AuthenticatedActor, "read", device_token: DeviceAccessToken)\n\tif has_role(actor, "external-authenticator", device_token.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", identity_provider: IdentityProvider)\n\tif has_role(actor, "external-authenticator", identity_provider.silo.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", saml_identity_provider: SamlIdentityProvider)\n\tif has_role(actor, "external-authenticator", saml_identity_provider.silo.fleet);\n\n# Describes the policy for who can access the internal database.\nresource Database {\n\tpermissions = [\n\t # "query" is required to perform any query against the database,\n\t # whether a read or write query. This is checked when an operation\n\t # checks out a database connection from the connection pool.\n\t #\n\t # Any authenticated user gets this permission. There's generally\n\t # some other authz check involved in the database query. For\n\t # example, if you're querying the database to "read" a "Project", we\n\t # should also be checking that. So why do we do this at all? It's\n\t # a belt-and-suspenders measure so that if we somehow introduced an\n\t # unauthenticated code path that hits the database, it cannot be\n\t # used to DoS the database because we won't allow the operation to\n\t # make the query. (As long as the code path _is_ authenticated, we\n\t # can use throttling mechanisms to prevent DoS.)\n\t "query",\n\n\t # "modify" is required to populate database data that's delivered\n\t # with the system. It should also be required for schema changes,\n\t # when we support those. This is separate from "query" so that we\n\t # cannot accidentally invoke these code paths from API calls and\n\t # other general functions.\n\t "modify"\n\t];\n}\n\n# All authenticated users have the "query" permission on the database.\nhas_permission(_actor: AuthenticatedActor, "query", _resource: Database);\n\n# The "db-init" user is the only one with the "modify" permission.\nhas_permission(USER_DB_INIT: AuthenticatedActor, "modify", _resource: Database);\nhas_permission(USER_DB_INIT: AuthenticatedActor, "create_child", _resource: IpPoolList);\n# It also has "admin" on the internal silo to populate it with built-in resources.\n# TODO-completeness: actually limit to just internal silo and not all silos\nhas_role(USER_DB_INIT: AuthenticatedActor, "admin", _silo: Silo);\n\n# Allow the internal API admin permissions on all silos.\nhas_role(USER_INTERNAL_API: AuthenticatedActor, "admin", _silo: Silo);\n\nresource WebhookSecret {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_webhook_receiver: WebhookReceiver };\n\n\t"read" if "read" on "parent_webhook_receiver";\n\t"modify" if "modify" on "parent_webhook_receiver";\n}\n\nhas_relation(rx: WebhookReceiver, "parent_webhook_receiver", secret: WebhookSecret)\n\tif secret.webhook_receiver = rx;\n\nresource WebhookEventClassList {\n\tpermissions = [ "list_children" ];\n\trelations = { parent_fleet: Fleet };\n\n\t"list_children" if "viewer" on "parent_fleet";\n}\n\nhas_relation(fleet: Fleet, "parent_fleet", collection: WebhookEventClassList)\n\tif collection.fleet = fleet;\n\n\n\n resource Disk {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Disk)\n if child.project = parent;\n \n\n resource Snapshot {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Snapshot)\n if child.project = parent;\n \n\n resource ProjectImage {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: ProjectImage)\n if child.project = parent;\n \n\n resource AffinityGroup {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: AffinityGroup)\n if child.project = parent;\n \n\n resource AntiAffinityGroup {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: AntiAffinityGroup)\n if child.project = parent;\n \n\n resource Instance {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Instance)\n if child.project = parent;\n \n\n resource IpPool {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: IpPool)\n if child.fleet = fleet;\n \n\n resource InstanceNetworkInterface {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Instance\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InstanceNetworkInterface)\n if has_relation(project, "containing_project", child.instance);\n\n has_relation(parent: Instance, "parent", child: InstanceNetworkInterface)\n if child.instance = parent;\n \n\n resource Vpc {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Vpc)\n if child.project = parent;\n \n\n resource VpcRouter {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Vpc\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: VpcRouter)\n if has_relation(project, "containing_project", child.vpc);\n\n has_relation(parent: Vpc, "parent", child: VpcRouter)\n if child.vpc = parent;\n \n\n resource InternetGateway {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Vpc\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InternetGateway)\n if has_relation(project, "containing_project", child.vpc);\n\n has_relation(parent: Vpc, "parent", child: InternetGateway)\n if child.vpc = parent;\n \n\n resource InternetGatewayIpPool {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: InternetGateway\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InternetGatewayIpPool)\n if has_relation(project, "containing_project", child.internet_gateway);\n\n has_relation(parent: InternetGateway, "parent", child: InternetGatewayIpPool)\n if child.internet_gateway = parent;\n \n\n resource InternetGatewayIpAddress {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: InternetGateway\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InternetGatewayIpAddress)\n if has_relation(project, "containing_project", child.internet_gateway);\n\n has_relation(parent: InternetGateway, "parent", child: InternetGatewayIpAddress)\n if child.internet_gateway = parent;\n \n\n resource RouterRoute {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: VpcRouter\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: RouterRoute)\n if has_relation(project, "containing_project", child.vpc_router);\n\n has_relation(parent: VpcRouter, "parent", child: RouterRoute)\n if child.vpc_router = parent;\n \n\n resource VpcSubnet {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Vpc\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: VpcSubnet)\n if has_relation(project, "containing_project", child.vpc);\n\n has_relation(parent: Vpc, "parent", child: VpcSubnet)\n if child.vpc = parent;\n \n\n resource FloatingIp {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: FloatingIp)\n if child.project = parent;\n \n\n resource Image {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_silo: Silo };\n "list_children" if "viewer" on "containing_silo";\n "read" if "viewer" on "containing_silo";\n "modify" if "collaborator" on "containing_silo";\n "create_child" if "collaborator" on "containing_silo";\n }\n\n has_relation(parent: Silo, "containing_silo", child: Image)\n if child.silo = parent;\n \n\n resource SiloImage {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_silo: Silo };\n "list_children" if "viewer" on "containing_silo";\n "read" if "viewer" on "containing_silo";\n "modify" if "collaborator" on "containing_silo";\n "create_child" if "collaborator" on "containing_silo";\n }\n\n has_relation(parent: Silo, "containing_silo", child: SiloImage)\n if child.silo = parent;\n \n\n resource AddressLot {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: AddressLot)\n if child.fleet = fleet;\n \n\n resource Blueprint {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Blueprint)\n if child.fleet = fleet;\n \n\n resource LoopbackAddress {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: LoopbackAddress)\n if child.fleet = fleet;\n \n\n\n resource ConsoleSession {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: ConsoleSession)\n if child.fleet = fleet;\n \n\n resource DeviceAuthRequest {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: DeviceAuthRequest)\n if child.fleet = fleet;\n \n\n resource DeviceAccessToken {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: DeviceAccessToken)\n if child.fleet = fleet;\n \n\n resource PhysicalDisk {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: PhysicalDisk)\n if child.fleet = fleet;\n \n\n resource Rack {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Rack)\n if child.fleet = fleet;\n \n\n resource RoleBuiltin {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: RoleBuiltin)\n if child.fleet = fleet;\n \n\n\n\n\n\n resource SupportBundle {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: SupportBundle)\n if child.fleet = fleet;\n \n\n\n\n resource Sled {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Sled)\n if child.fleet = fleet;\n \n\n resource TufRepo {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: TufRepo)\n if child.fleet = fleet;\n \n\n resource TufArtifact {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: TufArtifact)\n if child.fleet = fleet;\n \n\n resource WebhookEvent {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: WebhookEvent)\n if child.fleet = fleet;\n \n\n resource WebhookReceiver {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: WebhookReceiver)\n if child.fleet = fleet;\n \n\n\n resource Zpool {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Zpool)\n if child.fleet = fleet;\n \n\n resource Service {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Service)\n if child.fleet = fleet;\n \n\n resource UserBuiltin {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: UserBuiltin)\n if child.fleet = fleet;\n
1032025-04-30T06:14:00.822ZINFOtest_basic (ServerContext): Setting up resolver using DNS address: [::1]:64728
1042025-04-30T06:14:00.823ZINFOtest_basic (DnsResolver): new DNS resolver addresses = [[::1]:64728]
1052025-04-30T06:14:00.823ZINFOtest_basic (ServerContext): Setting up qorb database pool from a single host url = PostgresConfigWithUrl {\n url_raw: "postgresql://root@[::1]:47002/omicron?sslmode=disable",\n config: Config {\n user: Some(\n "root",\n ),\n password: None,\n dbname: Some(\n "omicron",\n ),\n options: None,\n application_name: None,\n ssl_mode: Disable,\n host: [\n Tcp(\n "::1",\n ),\n ],\n hostaddr: [],\n port: [\n 47002,\n ],\n connect_timeout: None,\n tcp_user_timeout: None,\n keepalives: true,\n keepalives_idle: 7200s,\n keepalives_interval: None,\n keepalives_retries: None,\n target_session_attrs: Any,\n channel_binding: Prefer,\n load_balance_hosts: Disable,\n },\n}
1062025-04-30T06:14:00.827ZDEBGtest_basic (ServerContext): registered USDT probes
1072025-04-30T06:14:01.008ZINFOtest_basic (nexus): Database schema version is up to date desired_version = 140.0.0 found_version = 140.0.0
1082025-04-30T06:14:01.146ZDEBGtest_basic (clickhouse-client): registered USDT probes id = 48b5afb2-831f-4ece-8e27-1dd2fcb099ce
1092025-04-30T06:14:01.240ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = GET path = /bgtasks versions = all
1102025-04-30T06:14:01.240ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = POST path = /bgtasks/activate versions = all
1112025-04-30T06:14:01.240ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = GET path = /bgtasks/view/{bgtask_name} versions = all
1122025-04-30T06:14:01.241ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = GET path = /clickhouse/policy versions = all
1132025-04-30T06:14:01.253ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = POST path = /clickhouse/policy versions = all
1142025-04-30T06:14:01.253ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = POST path = /crucible/0/upstairs/{upstairs_id}/downstairs/{downstairs_id}/stop-request versions = all
1152025-04-30T06:14:01.253ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = POST path = /crucible/0/upstairs/{upstairs_id}/downstairs/{downstairs_id}/stopped versions = all
1162025-04-30T06:14:01.253ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = POST path = /crucible/0/upstairs/{upstairs_id}/repair/{repair_id}/progress versions = all
1172025-04-30T06:14:01.253ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = POST path = /crucible/0/upstairs/{upstairs_id}/repair-finish versions = all
1182025-04-30T06:14:01.253ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = POST path = /crucible/0/upstairs/{upstairs_id}/repair-start versions = all
1192025-04-30T06:14:01.254ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = POST path = /demo-saga versions = all
1202025-04-30T06:14:01.254ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = POST path = /demo-saga/{demo_saga_id}/complete versions = all
1212025-04-30T06:14:01.254ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = GET path = /deployment/blueprints/all versions = all
1222025-04-30T06:14:01.254ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = DELETE path = /deployment/blueprints/all/{blueprint_id} versions = all
1232025-04-30T06:14:01.254ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = GET path = /deployment/blueprints/all/{blueprint_id} versions = all
1242025-04-30T06:14:01.254ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = POST path = /deployment/blueprints/import versions = all
1252025-04-30T06:14:01.254ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = POST path = /deployment/blueprints/regenerate versions = all
1262025-04-30T06:14:01.254ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = GET path = /deployment/blueprints/target versions = all
1272025-04-30T06:14:01.255ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = POST path = /deployment/blueprints/target versions = all
1282025-04-30T06:14:01.255ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = PUT path = /deployment/blueprints/target/enabled versions = all
1292025-04-30T06:14:01.255ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = POST path = /disk/{disk_id}/remove-read-only-parent versions = all
1302025-04-30T06:14:01.255ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = PUT path = /disks/{disk_id} versions = all
1312025-04-30T06:14:01.255ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = GET path = /experimental/v1/system/support-bundles versions = all
1322025-04-30T06:14:01.255ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = POST path = /experimental/v1/system/support-bundles versions = all
1332025-04-30T06:14:01.255ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = DELETE path = /experimental/v1/system/support-bundles/{support_bundle} versions = all
1342025-04-30T06:14:01.255ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = GET path = /experimental/v1/system/support-bundles/{support_bundle} versions = all
1352025-04-30T06:14:01.255ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = GET path = /experimental/v1/system/support-bundles/{support_bundle}/download versions = all
1362025-04-30T06:14:01.255ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = HEAD path = /experimental/v1/system/support-bundles/{support_bundle}/download versions = all
1372025-04-30T06:14:01.255ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = GET path = /experimental/v1/system/support-bundles/{support_bundle}/download/{file} versions = all
1382025-04-30T06:14:01.256ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = HEAD path = /experimental/v1/system/support-bundles/{support_bundle}/download/{file} versions = all
1392025-04-30T06:14:01.256ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = GET path = /experimental/v1/system/support-bundles/{support_bundle}/index versions = all
1402025-04-30T06:14:01.256ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = POST path = /instances/{instance_id}/migrate versions = all
1412025-04-30T06:14:01.256ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = POST path = /metrics/collectors versions = all
1422025-04-30T06:14:01.256ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = GET path = /metrics/collectors/{collector_id}/producers versions = all
1432025-04-30T06:14:01.256ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = POST path = /metrics/producers versions = all
1442025-04-30T06:14:01.257ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = GET path = /mgs-updates versions = all
1452025-04-30T06:14:01.257ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = GET path = /nat/ipv4/changeset/{from_gen} versions = all
1462025-04-30T06:14:01.257ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = GET path = /oximeter/read-policy versions = all
1472025-04-30T06:14:01.258ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = POST path = /oximeter/read-policy versions = all
1482025-04-30T06:14:01.258ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = POST path = /physical-disk/expunge versions = all
1492025-04-30T06:14:01.258ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = GET path = /probes/{sled} versions = all
1502025-04-30T06:14:01.258ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = PUT path = /racks/{rack_id}/initialization-complete versions = all
1512025-04-30T06:14:01.258ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = GET path = /sagas versions = all
1522025-04-30T06:14:01.258ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = GET path = /sagas/{saga_id} versions = all
1532025-04-30T06:14:01.258ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = GET path = /sled-agents/{sled_id} versions = all
1542025-04-30T06:14:01.258ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = POST path = /sled-agents/{sled_id} versions = all
1552025-04-30T06:14:01.258ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = POST path = /sled-agents/{sled_id}/firewall-rules-update versions = all
1562025-04-30T06:14:01.258ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = POST path = /sleds/add versions = all
1572025-04-30T06:14:01.258ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = POST path = /sleds/expunge versions = all
1582025-04-30T06:14:01.259ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = GET path = /sleds/uninitialized versions = all
1592025-04-30T06:14:01.259ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = PUT path = /switch/{switch_id} versions = all
1602025-04-30T06:14:01.259ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = GET path = /v1/ping versions = all
1612025-04-30T06:14:01.259ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = PUT path = /vmms/{propolis_id} versions = all
1622025-04-30T06:14:01.259ZDEBGtest_basic (dropshot_internal): registered endpoint local_addr = 127.0.0.1:34211 method = POST path = /volume/{volume_id}/remove-read-only-parent versions = all
1632025-04-30T06:14:01.260ZINFOtest_basic (dropshot_internal): listening local_addr = 127.0.0.1:34211
1642025-04-30T06:14:01.261ZDEBGtest_basic (dropshot_internal): successfully registered DTrace USDT probes local_addr = 127.0.0.1:34211
1652025-04-30T06:14:01.261ZINFOtest_basic (SEC): SEC running sec_id = 088ac96c-981b-4054-bea6-d8ab23ae1366
1662025-04-30T06:14:01.262ZTRACtest_basic (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1672025-04-30T06:14:01.262ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
1682025-04-30T06:14:01.265ZDEBGtest_basic (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1692025-04-30T06:14:01.266ZDEBGtest_basic (DataLoader): attempting to create built-in users
1702025-04-30T06:14:01.267ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1712025-04-30T06:14:01.267ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
1722025-04-30T06:14:01.268ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1732025-04-30T06:14:01.268ZINFOtest_basic (MgsUpdateDriver): starting MgsUpdateDriver
1742025-04-30T06:14:01.277ZTRACtest_basic (dns): buffer buffer = Length: 79 (0x4f) bytes\n0000: a8 dc 01 00 00 01 00 00 00 00 00 01 10 5f 63 72 ............._cr\n0010: 75 63 69 62 6c 65 2d 70 61 6e 74 72 79 04 5f 74 ucible-pantry._t\n0020: 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 cp.control-plane\n0030: 05 6f 78 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 .oxide.internal.\n0040: 00 21 00 01 00 00 29 04 d0 00 00 00 00 00 00 .!....)........ kind = dns peer_addr = [::1]:56404 req_id = 6136aabd-7cdc-4ace-a803-c040ca9a1d76
1752025-04-30T06:14:01.277ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 43228,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_crucible-pantry._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_crucible-pantry._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 16,\n 95,\n 99,\n 114,\n 117,\n 99,\n 105,\n 98,\n 108,\n 101,\n 45,\n 112,\n 97,\n 110,\n 116,\n 114,\n 121,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:56404 req_id = 6136aabd-7cdc-4ace-a803-c040ca9a1d76
1762025-04-30T06:14:01.280ZERROtest_basic (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 43228, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_crucible-pantry._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_crucible-pantry._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 16, 95, 99, 114, 117, 99, 105, 98, 108, 101, 45, 112, 97, 110, 116, 114, 121, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_crucible-pantry._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:56404 req_id = 6136aabd-7cdc-4ace-a803-c040ca9a1d76
1772025-04-30T06:14:01.286ZINFOtest_basic (DataLoader): created 5 built-in users
1782025-04-30T06:14:01.286ZTRACtest_basic (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1792025-04-30T06:14:01.286ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
1802025-04-30T06:14:01.289ZDEBGtest_basic (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1812025-04-30T06:14:01.289ZDEBGtest_basic (DataLoader): attempting to create built-in roles
1822025-04-30T06:14:01.289ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1832025-04-30T06:14:01.289ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
1842025-04-30T06:14:01.291ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1852025-04-30T06:14:01.304ZINFOtest_basic (DataLoader): created 10 built-in roles
1862025-04-30T06:14:01.304ZTRACtest_basic (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1872025-04-30T06:14:01.304ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
1882025-04-30T06:14:01.306ZDEBGtest_basic (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1892025-04-30T06:14:01.306ZDEBGtest_basic (DataLoader): attempting to create built-in role assignments
1902025-04-30T06:14:01.307ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1912025-04-30T06:14:01.307ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
1922025-04-30T06:14:01.307ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1932025-04-30T06:14:01.554ZINFOtest_basic (DataLoader): created 4 built-in role assignments
1942025-04-30T06:14:01.554ZTRACtest_basic (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1952025-04-30T06:14:01.555ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
1962025-04-30T06:14:01.557ZDEBGtest_basic (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1972025-04-30T06:14:01.557ZDEBGtest_basic (DataLoader): attempting to create built-in silos
1982025-04-30T06:14:01.557ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1992025-04-30T06:14:01.557ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
2002025-04-30T06:14:01.557ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2012025-04-30T06:14:01.658ZINFOtest_basic (DataLoader): created 2 built-in silos
2022025-04-30T06:14:01.658ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2032025-04-30T06:14:01.658ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
2042025-04-30T06:14:01.664ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2052025-04-30T06:14:01.684ZTRACtest_basic (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2062025-04-30T06:14:01.685ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
2072025-04-30T06:14:01.688ZDEBGtest_basic (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2082025-04-30T06:14:01.688ZDEBGtest_basic (DataLoader): attempting to create built-in projects
2092025-04-30T06:14:01.689ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2102025-04-30T06:14:01.690ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
2112025-04-30T06:14:01.691ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2122025-04-30T06:14:01.700ZTRACtest_basic (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }
2132025-04-30T06:14:01.700ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
2142025-04-30T06:14:01.700ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2152025-04-30T06:14:01.700ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
2162025-04-30T06:14:01.702ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2172025-04-30T06:14:01.767ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
2182025-04-30T06:14:01.767ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2192025-04-30T06:14:01.767ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
2202025-04-30T06:14:01.769ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2212025-04-30T06:14:01.776ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
2222025-04-30T06:14:01.781ZDEBGtest_basic (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) } result = Ok(())
2232025-04-30T06:14:01.781ZTRACtest_basic (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }
2242025-04-30T06:14:01.781ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
2252025-04-30T06:14:01.781ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2262025-04-30T06:14:01.781ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
2272025-04-30T06:14:01.782ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2282025-04-30T06:14:01.786ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
2292025-04-30T06:14:01.786ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2302025-04-30T06:14:01.786ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
2312025-04-30T06:14:01.787ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2322025-04-30T06:14:01.790ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
2332025-04-30T06:14:01.798ZDEBGtest_basic (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) } result = Ok(())
2342025-04-30T06:14:01.798ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2352025-04-30T06:14:01.798ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
2362025-04-30T06:14:01.800ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2372025-04-30T06:14:01.828ZINFOtest_basic (DataLoader): created built-in services project
2382025-04-30T06:14:01.828ZTRACtest_basic (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2392025-04-30T06:14:01.828ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
2402025-04-30T06:14:01.836ZDEBGtest_basic (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2412025-04-30T06:14:01.836ZDEBGtest_basic (DataLoader): attempting to create built-in VPCs
2422025-04-30T06:14:01.836ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2432025-04-30T06:14:01.836ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
2442025-04-30T06:14:01.837ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2452025-04-30T06:14:01.844ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2462025-04-30T06:14:01.844ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
2472025-04-30T06:14:01.845ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2482025-04-30T06:14:01.851ZTRACtest_basic (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }
2492025-04-30T06:14:01.851ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
2502025-04-30T06:14:01.851ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2512025-04-30T06:14:01.851ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
2522025-04-30T06:14:01.853ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2532025-04-30T06:14:01.864ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
2542025-04-30T06:14:01.864ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2552025-04-30T06:14:01.864ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
2562025-04-30T06:14:01.865ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2572025-04-30T06:14:01.883ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
2582025-04-30T06:14:01.883ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2592025-04-30T06:14:01.883ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
2602025-04-30T06:14:01.885ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2612025-04-30T06:14:01.888ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
2622025-04-30T06:14:01.898ZDEBGtest_basic (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) } result = Ok(())
2632025-04-30T06:14:01.898ZTRACtest_basic (DataLoader): successful lookup of siloed resource "Project" using built-in user
2642025-04-30T06:14:01.898ZTRACtest_basic (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }
2652025-04-30T06:14:01.899ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
2662025-04-30T06:14:01.899ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2672025-04-30T06:14:01.899ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
2682025-04-30T06:14:01.899ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2692025-04-30T06:14:01.903ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
2702025-04-30T06:14:01.903ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2712025-04-30T06:14:01.903ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
2722025-04-30T06:14:01.904ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2732025-04-30T06:14:01.908ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
2742025-04-30T06:14:01.908ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2752025-04-30T06:14:01.908ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
2762025-04-30T06:14:01.910ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2772025-04-30T06:14:01.914ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
2782025-04-30T06:14:01.928ZDEBGtest_basic (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) } result = Ok(())
2792025-04-30T06:14:01.928ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2802025-04-30T06:14:01.928ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
2812025-04-30T06:14:01.929ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2822025-04-30T06:14:01.954ZTRACtest_basic (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }
2832025-04-30T06:14:01.954ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
2842025-04-30T06:14:01.954ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2852025-04-30T06:14:01.954ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
2862025-04-30T06:14:01.956ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2872025-04-30T06:14:01.962ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
2882025-04-30T06:14:01.962ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2892025-04-30T06:14:01.962ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
2902025-04-30T06:14:01.963ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2912025-04-30T06:14:01.968ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
2922025-04-30T06:14:01.968ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2932025-04-30T06:14:01.968ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
2942025-04-30T06:14:01.970ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2952025-04-30T06:14:01.973ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
2962025-04-30T06:14:01.983ZDEBGtest_basic (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") } result = Ok(())
2972025-04-30T06:14:01.983ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2982025-04-30T06:14:01.983ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
2992025-04-30T06:14:01.985ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3002025-04-30T06:14:01.996ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3012025-04-30T06:14:01.996ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
3022025-04-30T06:14:01.997ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3032025-04-30T06:14:02.018ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3042025-04-30T06:14:02.019ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
3052025-04-30T06:14:02.019ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3062025-04-30T06:14:02.040ZTRACtest_basic (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }
3072025-04-30T06:14:02.040ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
3082025-04-30T06:14:02.041ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3092025-04-30T06:14:02.041ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
3102025-04-30T06:14:02.042ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3112025-04-30T06:14:02.057ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
3122025-04-30T06:14:02.058ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3132025-04-30T06:14:02.058ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
3142025-04-30T06:14:02.059ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3152025-04-30T06:14:02.238ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
3162025-04-30T06:14:02.238ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3172025-04-30T06:14:02.239ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
3182025-04-30T06:14:02.240ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3192025-04-30T06:14:02.245ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
3202025-04-30T06:14:02.253ZDEBGtest_basic (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") } result = Ok(())
3212025-04-30T06:14:02.254ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3222025-04-30T06:14:02.254ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
3232025-04-30T06:14:02.256ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3242025-04-30T06:14:02.312ZTRACtest_basic (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) }
3252025-04-30T06:14:02.312ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
3262025-04-30T06:14:02.312ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3272025-04-30T06:14:02.312ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
3282025-04-30T06:14:02.313ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3292025-04-30T06:14:02.317ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
3302025-04-30T06:14:02.317ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3312025-04-30T06:14:02.317ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
3322025-04-30T06:14:02.319ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3332025-04-30T06:14:02.324ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
3342025-04-30T06:14:02.324ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3352025-04-30T06:14:02.324ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
3362025-04-30T06:14:02.325ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3372025-04-30T06:14:02.328ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
3382025-04-30T06:14:02.337ZDEBGtest_basic (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
3392025-04-30T06:14:02.337ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3402025-04-30T06:14:02.337ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
3412025-04-30T06:14:02.337ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3422025-04-30T06:14:02.430ZTRACtest_basic (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) }
3432025-04-30T06:14:02.430ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
3442025-04-30T06:14:02.431ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3452025-04-30T06:14:02.431ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
3462025-04-30T06:14:02.431ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3472025-04-30T06:14:02.434ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
3482025-04-30T06:14:02.434ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3492025-04-30T06:14:02.434ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
3502025-04-30T06:14:02.435ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3512025-04-30T06:14:02.437ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
3522025-04-30T06:14:02.437ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3532025-04-30T06:14:02.437ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
3542025-04-30T06:14:02.438ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3552025-04-30T06:14:02.441ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
3562025-04-30T06:14:02.448ZDEBGtest_basic (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
3572025-04-30T06:14:02.448ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3582025-04-30T06:14:02.448ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
3592025-04-30T06:14:02.450ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3602025-04-30T06:14:02.460ZDEBGtest_basic (DataLoader): attempting to create built-in VPC firewall rules
3612025-04-30T06:14:02.460ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3622025-04-30T06:14:02.460ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
3632025-04-30T06:14:02.461ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3642025-04-30T06:14:02.467ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3652025-04-30T06:14:02.467ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
3662025-04-30T06:14:02.468ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3672025-04-30T06:14:02.470ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3682025-04-30T06:14:02.470ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
3692025-04-30T06:14:02.470ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3702025-04-30T06:14:02.472ZTRACtest_basic (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
3712025-04-30T06:14:02.472ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
3722025-04-30T06:14:02.472ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3732025-04-30T06:14:02.472ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
3742025-04-30T06:14:02.473ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3752025-04-30T06:14:02.478ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
3762025-04-30T06:14:02.478ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3772025-04-30T06:14:02.478ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
3782025-04-30T06:14:02.480ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3792025-04-30T06:14:02.485ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
3802025-04-30T06:14:02.485ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3812025-04-30T06:14:02.485ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
3822025-04-30T06:14:02.486ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3832025-04-30T06:14:02.491ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
3842025-04-30T06:14:02.501ZDEBGtest_basic (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
3852025-04-30T06:14:02.501ZTRACtest_basic (DataLoader): successful lookup of siloed resource "Vpc" using built-in user
3862025-04-30T06:14:02.502ZTRACtest_basic (DataLoader): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
3872025-04-30T06:14:02.502ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
3882025-04-30T06:14:02.502ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3892025-04-30T06:14:02.502ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
3902025-04-30T06:14:02.508ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3912025-04-30T06:14:02.511ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
3922025-04-30T06:14:02.511ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3932025-04-30T06:14:02.511ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
3942025-04-30T06:14:02.512ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3952025-04-30T06:14:02.516ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
3962025-04-30T06:14:02.516ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3972025-04-30T06:14:02.516ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
3982025-04-30T06:14:02.517ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3992025-04-30T06:14:02.520ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
4002025-04-30T06:14:02.529ZDEBGtest_basic (DataLoader): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
4012025-04-30T06:14:02.529ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4022025-04-30T06:14:02.529ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
4032025-04-30T06:14:02.530ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4042025-04-30T06:14:02.575ZTRACtest_basic (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
4052025-04-30T06:14:02.575ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
4062025-04-30T06:14:02.576ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4072025-04-30T06:14:02.576ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
4082025-04-30T06:14:02.577ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4092025-04-30T06:14:02.584ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
4102025-04-30T06:14:02.584ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4112025-04-30T06:14:02.584ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
4122025-04-30T06:14:02.585ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4132025-04-30T06:14:02.588ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
4142025-04-30T06:14:02.588ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4152025-04-30T06:14:02.588ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
4162025-04-30T06:14:02.589ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4172025-04-30T06:14:02.594ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
4182025-04-30T06:14:02.601ZDEBGtest_basic (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
4192025-04-30T06:14:02.601ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4202025-04-30T06:14:02.601ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
4212025-04-30T06:14:02.602ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4222025-04-30T06:14:02.700ZINFOtest_basic (DataLoader): created built-in VPC firewall rules
4232025-04-30T06:14:02.700ZDEBGtest_basic (DataLoader): attempting to create built-in VPC Subnets
4242025-04-30T06:14:02.700ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4252025-04-30T06:14:02.700ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
4262025-04-30T06:14:02.700ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4272025-04-30T06:14:02.704ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4282025-04-30T06:14:02.704ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
4292025-04-30T06:14:02.706ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4302025-04-30T06:14:02.708ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4312025-04-30T06:14:02.709ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
4322025-04-30T06:14:02.709ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4332025-04-30T06:14:02.712ZTRACtest_basic (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
4342025-04-30T06:14:02.712ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
4352025-04-30T06:14:02.712ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4362025-04-30T06:14:02.712ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
4372025-04-30T06:14:02.712ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4382025-04-30T06:14:02.714ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
4392025-04-30T06:14:02.714ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4402025-04-30T06:14:02.715ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
4412025-04-30T06:14:02.715ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4422025-04-30T06:14:02.717ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
4432025-04-30T06:14:02.717ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4442025-04-30T06:14:02.717ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
4452025-04-30T06:14:02.718ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4462025-04-30T06:14:02.720ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
4472025-04-30T06:14:02.725ZDEBGtest_basic (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
4482025-04-30T06:14:02.725ZTRACtest_basic (DataLoader): successful lookup of siloed resource "Vpc" using built-in user
4492025-04-30T06:14:02.725ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4502025-04-30T06:14:02.725ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
4512025-04-30T06:14:02.726ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4522025-04-30T06:14:02.733ZTRACtest_basic (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
4532025-04-30T06:14:02.733ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
4542025-04-30T06:14:02.733ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4552025-04-30T06:14:02.733ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
4562025-04-30T06:14:02.733ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4572025-04-30T06:14:02.735ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
4582025-04-30T06:14:02.735ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4592025-04-30T06:14:02.735ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
4602025-04-30T06:14:02.736ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4612025-04-30T06:14:02.738ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
4622025-04-30T06:14:02.738ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4632025-04-30T06:14:02.738ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
4642025-04-30T06:14:02.739ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4652025-04-30T06:14:02.741ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
4662025-04-30T06:14:02.746ZDEBGtest_basic (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
4672025-04-30T06:14:02.753ZTRACtest_basic (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) }
4682025-04-30T06:14:02.753ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
4692025-04-30T06:14:02.754ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4702025-04-30T06:14:02.754ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
4712025-04-30T06:14:02.754ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4722025-04-30T06:14:02.758ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
4732025-04-30T06:14:02.759ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4742025-04-30T06:14:02.759ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
4752025-04-30T06:14:02.760ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4762025-04-30T06:14:02.763ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
4772025-04-30T06:14:02.763ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4782025-04-30T06:14:02.763ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
4792025-04-30T06:14:02.764ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4802025-04-30T06:14:02.767ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
4812025-04-30T06:14:02.774ZDEBGtest_basic (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
4822025-04-30T06:14:02.774ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4832025-04-30T06:14:02.774ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
4842025-04-30T06:14:02.775ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4852025-04-30T06:14:02.825ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4862025-04-30T06:14:02.825ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
4872025-04-30T06:14:02.826ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4882025-04-30T06:14:02.829ZTRACtest_basic (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
4892025-04-30T06:14:02.829ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
4902025-04-30T06:14:02.829ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4912025-04-30T06:14:02.829ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
4922025-04-30T06:14:02.830ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4932025-04-30T06:14:02.832ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
4942025-04-30T06:14:02.833ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4952025-04-30T06:14:02.833ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
4962025-04-30T06:14:02.833ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4972025-04-30T06:14:02.836ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
4982025-04-30T06:14:02.836ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4992025-04-30T06:14:02.836ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
5002025-04-30T06:14:02.838ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5012025-04-30T06:14:02.841ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
5022025-04-30T06:14:02.860ZDEBGtest_basic (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
5032025-04-30T06:14:02.873ZTRACtest_basic (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) }
5042025-04-30T06:14:02.874ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
5052025-04-30T06:14:02.874ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5062025-04-30T06:14:02.874ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
5072025-04-30T06:14:02.875ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5082025-04-30T06:14:02.879ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
5092025-04-30T06:14:02.879ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5102025-04-30T06:14:02.879ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
5112025-04-30T06:14:02.880ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5122025-04-30T06:14:02.884ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
5132025-04-30T06:14:02.884ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5142025-04-30T06:14:02.884ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
5152025-04-30T06:14:02.885ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5162025-04-30T06:14:02.888ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
5172025-04-30T06:14:02.903ZDEBGtest_basic (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
5182025-04-30T06:14:02.904ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5192025-04-30T06:14:02.904ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
5202025-04-30T06:14:02.905ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5212025-04-30T06:14:02.914ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5222025-04-30T06:14:02.914ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
5232025-04-30T06:14:02.917ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5242025-04-30T06:14:02.923ZTRACtest_basic (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
5252025-04-30T06:14:02.923ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
5262025-04-30T06:14:02.923ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5272025-04-30T06:14:02.923ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
5282025-04-30T06:14:02.924ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5292025-04-30T06:14:02.927ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
5302025-04-30T06:14:02.927ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5312025-04-30T06:14:02.927ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
5322025-04-30T06:14:02.928ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5332025-04-30T06:14:02.934ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
5342025-04-30T06:14:02.934ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5352025-04-30T06:14:02.934ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
5362025-04-30T06:14:02.934ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5372025-04-30T06:14:02.937ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
5382025-04-30T06:14:02.945ZDEBGtest_basic (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
5392025-04-30T06:14:02.959ZTRACtest_basic (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) }
5402025-04-30T06:14:02.960ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
5412025-04-30T06:14:02.960ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5422025-04-30T06:14:02.960ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
5432025-04-30T06:14:02.962ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5442025-04-30T06:14:02.976ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
5452025-04-30T06:14:02.977ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5462025-04-30T06:14:02.983ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
5472025-04-30T06:14:02.987ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5482025-04-30T06:14:02.997ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
5492025-04-30T06:14:02.997ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5502025-04-30T06:14:02.997ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
5512025-04-30T06:14:02.999ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5522025-04-30T06:14:03.005ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
5532025-04-30T06:14:03.029ZDEBGtest_basic (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
5542025-04-30T06:14:03.030ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5552025-04-30T06:14:03.030ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
5562025-04-30T06:14:03.032ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5572025-04-30T06:14:03.050ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5582025-04-30T06:14:03.050ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
5592025-04-30T06:14:03.052ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5602025-04-30T06:14:03.059ZINFOtest_basic (DataLoader): created built-in services vpc subnets
5612025-04-30T06:14:03.059ZINFOtest_basic (DataLoader): created built-in services vpc
5622025-04-30T06:14:03.060ZTRACtest_basic (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5632025-04-30T06:14:03.060ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
5642025-04-30T06:14:03.065ZDEBGtest_basic (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5652025-04-30T06:14:03.065ZDEBGtest_basic (DataLoader): attempting to create silo users
5662025-04-30T06:14:03.065ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5672025-04-30T06:14:03.065ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
5682025-04-30T06:14:03.066ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5692025-04-30T06:14:03.075ZINFOtest_basic (DataLoader): created 2 silo users
5702025-04-30T06:14:03.075ZDEBGtest_basic (DataLoader): attempting to create silo user role assignments
5712025-04-30T06:14:03.075ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5722025-04-30T06:14:03.077ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
5732025-04-30T06:14:03.080ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5742025-04-30T06:14:03.087ZINFOtest_basic (DataLoader): created 2 silo user role assignments
5752025-04-30T06:14:03.087ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5762025-04-30T06:14:03.087ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
5772025-04-30T06:14:03.088ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5782025-04-30T06:14:03.109ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5792025-04-30T06:14:03.110ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
5802025-04-30T06:14:03.111ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5812025-04-30T06:14:03.139ZTRACtest_basic (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = IpPoolList
5822025-04-30T06:14:03.139ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
5832025-04-30T06:14:03.139ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5842025-04-30T06:14:03.139ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
5852025-04-30T06:14:03.141ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5862025-04-30T06:14:03.151ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
5872025-04-30T06:14:03.155ZDEBGtest_basic (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = IpPoolList result = Ok(())
5882025-04-30T06:14:03.156ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5892025-04-30T06:14:03.156ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
5902025-04-30T06:14:03.156ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5912025-04-30T06:14:03.176ZTRACtest_basic (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = IpPoolList
5922025-04-30T06:14:03.176ZTRACtest_basic (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
5932025-04-30T06:14:03.176ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5942025-04-30T06:14:03.176ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
5952025-04-30T06:14:03.177ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5962025-04-30T06:14:03.189ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
5972025-04-30T06:14:03.192ZDEBGtest_basic (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = IpPoolList result = Ok(())
5982025-04-30T06:14:03.192ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5992025-04-30T06:14:03.192ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
6002025-04-30T06:14:03.192ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
6012025-04-30T06:14:03.334ZTRACtest_basic (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
6022025-04-30T06:14:03.334ZDEBGtest_basic (DataLoader): roles roles = RoleSet { roles: {} }
6032025-04-30T06:14:03.335ZDEBGtest_basic (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
6042025-04-30T06:14:03.338ZINFOtest_basic (nexus): populate complete
6052025-04-30T06:14:03.338ZINFOtest_basic (nexus): activating background tasks
6062025-04-30T06:14:03.342ZDEBGtest_basic (init_with_steps): Running step start_sled1
6072025-04-30T06:14:03.342ZINFOtest_basic (omicron_sled_agent::sim::Server): setting up sled agent server sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6082025-04-30T06:14:03.433ZINFOtest_basic (SledAgent): created simulated sled agent server = b6d65341-167c-41df-9b5c-41cded99c229 sim_mode = Explicit sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6092025-04-30T06:14:03.433ZINFOtest_basic (omicron_sled_agent::sim::SimulatedUpstairs): registered sled b6d65341-167c-41df-9b5c-41cded99c229 storage
6102025-04-30T06:14:03.434ZDEBGtest_basic (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 1 reason = Timeout
6112025-04-30T06:14:03.434ZDEBGtest_basic (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
6122025-04-30T06:14:03.434ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
6132025-04-30T06:14:03.434ZDEBGtest_basic (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6142025-04-30T06:14:03.435ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6152025-04-30T06:14:03.435ZDEBGtest_basic (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 1 reason = Timeout
6162025-04-30T06:14:03.435ZTRACtest_basic (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal
6172025-04-30T06:14:03.435ZDEBGtest_basic (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 1 reason = Timeout
6182025-04-30T06:14:03.435ZWARNtest_basic (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_internal dns_group = internal reason = no config nor servers
6192025-04-30T06:14:03.435ZDEBGtest_basic (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 96.613\u{b5}s iteration = 1 status = {"error":"no config nor servers"}
6202025-04-30T06:14:03.435ZDEBGtest_basic (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 1 reason = Timeout
6212025-04-30T06:14:03.435ZDEBGtest_basic (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
6222025-04-30T06:14:03.435ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
6232025-04-30T06:14:03.435ZDEBGtest_basic (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6242025-04-30T06:14:03.436ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6252025-04-30T06:14:03.436ZDEBGtest_basic (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 1 reason = Timeout
6262025-04-30T06:14:03.436ZTRACtest_basic (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal
6272025-04-30T06:14:03.436ZDEBGtest_basic (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 1 reason = Timeout
6282025-04-30T06:14:03.436ZWARNtest_basic (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_external dns_group = external reason = no config nor servers
6292025-04-30T06:14:03.436ZDEBGtest_basic (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 52.094\u{b5}s iteration = 1 status = {"error":"no config nor servers"}
6302025-04-30T06:14:03.436ZDEBGtest_basic (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 1 reason = Timeout
6312025-04-30T06:14:03.436ZINFOtest_basic (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-04-30 06:04:03.436507939 UTC
6322025-04-30T06:14:03.436ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database
6332025-04-30T06:14:03.436ZDEBGtest_basic (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
6342025-04-30T06:14:03.437ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
6352025-04-30T06:14:03.437ZDEBGtest_basic (BackgroundTasks): activating background_task = external_endpoints iteration = 1 reason = Timeout
6362025-04-30T06:14:03.437ZTRACtest_basic (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet
6372025-04-30T06:14:03.437ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = external_endpoints resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
6382025-04-30T06:14:03.437ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
6392025-04-30T06:14:03.437ZDEBGtest_basic (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6402025-04-30T06:14:03.437ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6412025-04-30T06:14:03.438ZDEBGtest_basic (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 1 reason = Timeout
6422025-04-30T06:14:03.438ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database
6432025-04-30T06:14:03.438ZDEBGtest_basic (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
6442025-04-30T06:14:03.438ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
6452025-04-30T06:14:03.438ZDEBGtest_basic (BackgroundTasks): activating background_task = bfd_manager iteration = 1 reason = Timeout
6462025-04-30T06:14:03.438ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database
6472025-04-30T06:14:03.438ZDEBGtest_basic (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
6482025-04-30T06:14:03.439ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
6492025-04-30T06:14:03.439ZDEBGtest_basic (BackgroundTasks): activating background_task = phantom_disks iteration = 1 reason = Timeout
6502025-04-30T06:14:03.439ZDEBGtest_basic (BackgroundTasks): activating background_task = blueprint_loader iteration = 1 reason = Timeout
6512025-04-30T06:14:03.439ZTRACtest_basic (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig
6522025-04-30T06:14:03.439ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = blueprint_loader resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
6532025-04-30T06:14:03.439ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database
6542025-04-30T06:14:03.439ZDEBGtest_basic (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6552025-04-30T06:14:03.439ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6562025-04-30T06:14:03.439ZDEBGtest_basic (BackgroundTasks): activating background_task = blueprint_executor iteration = 1 reason = Timeout
6572025-04-30T06:14:03.439ZWARNtest_basic (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor reason = no blueprint
6582025-04-30T06:14:03.439ZDEBGtest_basic (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 39.478\u{b5}s iteration = 1 status = {"error":"no blueprint"}
6592025-04-30T06:14:03.439ZDEBGtest_basic (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 1 reason = Timeout
6602025-04-30T06:14:03.439ZWARNtest_basic (BackgroundTasks): Blueprint execution: skipped background_task = crdb_node_id_collector reason = no blueprint
6612025-04-30T06:14:03.439ZDEBGtest_basic (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 17.235\u{b5}s iteration = 1 status = {"error":"no blueprint"}
6622025-04-30T06:14:03.439ZDEBGtest_basic (BackgroundTasks): activating background_task = inventory_collection iteration = 1 reason = Timeout
6632025-04-30T06:14:03.439ZTRACtest_basic (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
6642025-04-30T06:14:03.439ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
6652025-04-30T06:14:03.440ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
6662025-04-30T06:14:03.440ZDEBGtest_basic (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
6672025-04-30T06:14:03.440ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
6682025-04-30T06:14:03.440ZDEBGtest_basic (BackgroundTasks): activating background_task = support_bundle_collector iteration = 1 reason = Timeout
6692025-04-30T06:14:03.440ZTRACtest_basic (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet
6702025-04-30T06:14:03.440ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = support_bundle_collector resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
6712025-04-30T06:14:03.440ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database
6722025-04-30T06:14:03.440ZDEBGtest_basic (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
6732025-04-30T06:14:03.440ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
6742025-04-30T06:14:03.441ZDEBGtest_basic (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 1 reason = Timeout
6752025-04-30T06:14:03.441ZDEBGtest_basic (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 20.611\u{b5}s iteration = 1 status = {"error":"task disabled"}
6762025-04-30T06:14:03.441ZDEBGtest_basic (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 1 reason = Timeout
6772025-04-30T06:14:03.441ZWARNtest_basic (BackgroundTasks): Blueprint rendezvous: skipped background_task = blueprint_rendezvous reason = no blueprint
6782025-04-30T06:14:03.441ZDEBGtest_basic (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 42.735\u{b5}s iteration = 1 status = {"error":"no blueprint"}
6792025-04-30T06:14:03.441ZDEBGtest_basic (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 1 reason = Timeout
6802025-04-30T06:14:03.441ZINFOtest_basic (BackgroundTasks): Decommissioned Disk Cleaner disabled background_task = decommissioned_disk_cleaner
6812025-04-30T06:14:03.441ZDEBGtest_basic (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 219.948\u{b5}s iteration = 1 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
6822025-04-30T06:14:03.441ZDEBGtest_basic (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 1 reason = Timeout
6832025-04-30T06:14:03.441ZTRACtest_basic (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory
6842025-04-30T06:14:03.441ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_zone_nat_tracker resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
6852025-04-30T06:14:03.441ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database
6862025-04-30T06:14:03.441ZDEBGtest_basic (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6872025-04-30T06:14:03.443ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6882025-04-30T06:14:03.443ZDEBGtest_basic (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 1 reason = Timeout
6892025-04-30T06:14:03.443ZTRACtest_basic (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet
6902025-04-30T06:14:03.443ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = switch_port_config_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
6912025-04-30T06:14:03.443ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database
6922025-04-30T06:14:03.443ZDEBGtest_basic (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6932025-04-30T06:14:03.444ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6942025-04-30T06:14:03.445ZDEBGtest_basic (BackgroundTasks): activating background_task = v2p_manager iteration = 1 reason = Timeout
6952025-04-30T06:14:03.446ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
6962025-04-30T06:14:03.446ZDEBGtest_basic (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6972025-04-30T06:14:03.447ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6982025-04-30T06:14:03.447ZDEBGtest_basic (BackgroundTasks): activating background_task = region_replacement iteration = 1 reason = Timeout
6992025-04-30T06:14:03.447ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database
7002025-04-30T06:14:03.447ZDEBGtest_basic (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
7012025-04-30T06:14:03.448ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
7022025-04-30T06:14:03.448ZDEBGtest_basic (BackgroundTasks): activating background_task = region_replacement_driver iteration = 1 reason = Timeout
7032025-04-30T06:14:03.448ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database
7042025-04-30T06:14:03.448ZDEBGtest_basic (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
7052025-04-30T06:14:03.449ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7062025-04-30T06:14:03.449ZDEBGtest_basic (BackgroundTasks): activating background_task = instance_watcher iteration = 1 reason = Timeout
7072025-04-30T06:14:03.449ZTRACtest_basic (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet
7082025-04-30T06:14:03.450ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = instance_watcher resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
7092025-04-30T06:14:03.450ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database
7102025-04-30T06:14:03.451ZDEBGtest_basic (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
7112025-04-30T06:14:03.453ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
7122025-04-30T06:14:03.454ZDEBGtest_basic (BackgroundTasks): activating background_task = instance_updater iteration = 1 reason = Timeout
7132025-04-30T06:14:03.454ZINFOtest_basic (BackgroundTasks): background instance updater explicitly disabled background_task = instance_updater
7142025-04-30T06:14:03.455ZDEBGtest_basic (BackgroundTasks): activation complete background_task = instance_updater elapsed = 1.065323ms iteration = 1 status = {"destroyed_active_vmms":0,"disabled":true,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
7152025-04-30T06:14:03.455ZDEBGtest_basic (BackgroundTasks): activating background_task = instance_reincarnation iteration = 1 reason = Timeout
7162025-04-30T06:14:03.456ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database
7172025-04-30T06:14:03.456ZDEBGtest_basic (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
7182025-04-30T06:14:03.458ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
7192025-04-30T06:14:03.458ZDEBGtest_basic (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 1 reason = Timeout
7202025-04-30T06:14:03.458ZDEBGtest_basic (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
7212025-04-30T06:14:03.458ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
7222025-04-30T06:14:03.458ZDEBGtest_basic (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7232025-04-30T06:14:03.460ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7242025-04-30T06:14:03.461ZDEBGtest_basic (BackgroundTasks): activating background_task = vpc_route_manager iteration = 1 reason = Timeout
7252025-04-30T06:14:03.461ZINFOtest_basic (BackgroundTasks): VPC route manager running background_task = vpc_route_manager
7262025-04-30T06:14:03.461ZTRACtest_basic (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet
7272025-04-30T06:14:03.461ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = vpc_route_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
7282025-04-30T06:14:03.462ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
7292025-04-30T06:14:03.462ZDEBGtest_basic (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7302025-04-30T06:14:03.463ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7312025-04-30T06:14:03.463ZDEBGtest_basic (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 1 reason = Timeout
7322025-04-30T06:14:03.463ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database
7332025-04-30T06:14:03.463ZDEBGtest_basic (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
7342025-04-30T06:14:03.464ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
7352025-04-30T06:14:03.464ZDEBGtest_basic (BackgroundTasks): activating background_task = saga_recovery iteration = 1 reason = Timeout
7362025-04-30T06:14:03.464ZDEBGtest_basic (SagaRecoverer): listing candidate sagas for recovery
7372025-04-30T06:14:03.464ZTRACtest_basic (SagaRecoverer): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000003a8a, .. }) resource = Database
7382025-04-30T06:14:03.464ZDEBGtest_basic (SagaRecoverer): roles roles = RoleSet { roles: {} }
7392025-04-30T06:14:03.465ZDEBGtest_basic (SagaRecoverer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000003a8a, .. }) resource = Database result = Ok(())
7402025-04-30T06:14:03.465ZDEBGtest_basic (BackgroundTasks): activating background_task = lookup_region_port iteration = 1 reason = Timeout
7412025-04-30T06:14:03.465ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database
7422025-04-30T06:14:03.465ZDEBGtest_basic (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
7432025-04-30T06:14:03.466ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
7442025-04-30T06:14:03.466ZDEBGtest_basic (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 1 reason = Timeout
7452025-04-30T06:14:03.466ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database
7462025-04-30T06:14:03.466ZDEBGtest_basic (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
7472025-04-30T06:14:03.467ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
7482025-04-30T06:14:03.467ZDEBGtest_basic (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 1 reason = Timeout
7492025-04-30T06:14:03.467ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database
7502025-04-30T06:14:03.467ZDEBGtest_basic (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
7512025-04-30T06:14:03.467ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
7522025-04-30T06:14:03.468ZDEBGtest_basic (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 1 reason = Timeout
7532025-04-30T06:14:03.468ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database
7542025-04-30T06:14:03.468ZDEBGtest_basic (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
7552025-04-30T06:14:03.470ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
7562025-04-30T06:14:03.470ZDEBGtest_basic (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 1 reason = Timeout
7572025-04-30T06:14:03.470ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database
7582025-04-30T06:14:03.470ZDEBGtest_basic (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
7592025-04-30T06:14:03.470ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
7602025-04-30T06:14:03.470ZDEBGtest_basic (BackgroundTasks): activating background_task = tuf_artifact_replication iteration = 1 reason = Timeout
7612025-04-30T06:14:03.473ZTRACtest_basic (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet
7622025-04-30T06:14:03.489ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = tuf_artifact_replication resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
7632025-04-30T06:14:03.489ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database
7642025-04-30T06:14:03.489ZDEBGtest_basic (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
7652025-04-30T06:14:03.489ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
7662025-04-30T06:14:03.489ZDEBGtest_basic (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 1 reason = Timeout
7672025-04-30T06:14:03.489ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database
7682025-04-30T06:14:03.489ZDEBGtest_basic (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
7692025-04-30T06:14:03.489ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
7702025-04-30T06:14:03.489ZDEBGtest_basic (BackgroundTasks): activating background_task = webhook_dispatcher iteration = 1 reason = Timeout
7712025-04-30T06:14:03.489ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_dispatcher resource = Database
7722025-04-30T06:14:03.489ZDEBGtest_basic (BackgroundTasks): roles background_task = webhook_dispatcher roles = RoleSet { roles: {} }
7732025-04-30T06:14:03.489ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_dispatcher resource = Database result = Ok(())
7742025-04-30T06:14:03.489ZDEBGtest_basic (BackgroundTasks): activating background_task = webhook_deliverator iteration = 1 reason = Timeout
7752025-04-30T06:14:03.489ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database
7762025-04-30T06:14:03.489ZDEBGtest_basic (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
7772025-04-30T06:14:03.489ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
7782025-04-30T06:14:03.489ZINFOtest_basic (ArtifactStore): No ledger in /var/tmp/omicron_tmp/.tmppTSgOY/artifact-config.json server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
7792025-04-30T06:14:03.489ZDEBGtest_basic (ArtifactStore): Failed to read ledger: Not found in storage path = /var/tmp/omicron_tmp/.tmppTSgOY/artifact-config.json server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
7802025-04-30T06:14:03.489ZINFOtest_basic (ArtifactStore): No ledger in /var/tmp/omicron_tmp/.tmp7hnHjr/artifact-config.json server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
7812025-04-30T06:14:03.489ZDEBGtest_basic (ArtifactStore): Failed to read ledger: Not found in storage path = /var/tmp/omicron_tmp/.tmp7hnHjr/artifact-config.json server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
7822025-04-30T06:14:03.542ZDEBGtest_basic (dropshot (Repo Depot)): registered endpoint local_addr = [::1]:51632 method = GET path = /artifact/sha256/{sha256} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
7832025-04-30T06:14:03.542ZINFOtest_basic (dropshot (Repo Depot)): listening local_addr = [::1]:51632 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
7842025-04-30T06:14:03.543ZDEBGtest_basic (dropshot (Repo Depot)): successfully registered DTrace USDT probes local_addr = [::1]:51632 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
7852025-04-30T06:14:03.550ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = GET path = /artifacts sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
7862025-04-30T06:14:03.550ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = PUT path = /artifacts/{sha256} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
7872025-04-30T06:14:03.550ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = POST path = /artifacts/{sha256}/copy-from-depot sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
7882025-04-30T06:14:03.550ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = GET path = /artifacts-config sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
7892025-04-30T06:14:03.550ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = PUT path = /artifacts-config sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
7902025-04-30T06:14:03.550ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = POST path = /boot-disk/{boot_disk}/os/write sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
7912025-04-30T06:14:03.550ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = GET path = /boot-disk/{boot_disk}/os/write/status sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
7922025-04-30T06:14:03.550ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = DELETE path = /boot-disk/{boot_disk}/os/write/status/{update_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
7932025-04-30T06:14:03.550ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = GET path = /bootstore/status sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
7942025-04-30T06:14:03.550ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = GET path = /datasets sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
7952025-04-30T06:14:03.550ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = PUT path = /disks/{disk_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
7962025-04-30T06:14:03.550ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = POST path = /disks/{disk_id}/poke sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
7972025-04-30T06:14:03.550ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = PUT path = /eip-gateways sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
7982025-04-30T06:14:03.551ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = GET path = /inventory sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
7992025-04-30T06:14:03.551ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = GET path = /network-bootstore-config sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8002025-04-30T06:14:03.551ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = PUT path = /network-bootstore-config sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8012025-04-30T06:14:03.551ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = PUT path = /omicron-config sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8022025-04-30T06:14:03.551ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = GET path = /omicron-physical-disks sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8032025-04-30T06:14:03.551ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = GET path = /sled-identifiers sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8042025-04-30T06:14:03.551ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = GET path = /sled-role sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8052025-04-30T06:14:03.551ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = PUT path = /sleds sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8062025-04-30T06:14:03.551ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = GET path = /support/dladm-info sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8072025-04-30T06:14:03.551ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = GET path = /support/ipadm-info sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8082025-04-30T06:14:03.551ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = GET path = /support/logs/download/{zone} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8092025-04-30T06:14:03.551ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = GET path = /support/logs/zones sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8102025-04-30T06:14:03.551ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = GET path = /support/nvmeadm-info sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8112025-04-30T06:14:03.551ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = GET path = /support/pargs-info sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8122025-04-30T06:14:03.551ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = GET path = /support/pfiles-info sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8132025-04-30T06:14:03.551ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = GET path = /support/pstack-info sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8142025-04-30T06:14:03.551ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = GET path = /support/zfs-info sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8152025-04-30T06:14:03.551ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = GET path = /support/zoneadm-info sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8162025-04-30T06:14:03.551ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = GET path = /support/zpool-info sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8172025-04-30T06:14:03.551ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = GET path = /support-bundles/{zpool_id}/{dataset_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8182025-04-30T06:14:03.551ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = DELETE path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8192025-04-30T06:14:03.551ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = POST path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8202025-04-30T06:14:03.551ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = GET path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id}/download sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8212025-04-30T06:14:03.551ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = HEAD path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id}/download sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8222025-04-30T06:14:03.551ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = GET path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id}/download/{file} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8232025-04-30T06:14:03.551ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = HEAD path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id}/download/{file} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8242025-04-30T06:14:03.551ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = GET path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id}/index sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8252025-04-30T06:14:03.551ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = HEAD path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id}/index sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8262025-04-30T06:14:03.551ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = POST path = /switch-ports sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8272025-04-30T06:14:03.551ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = GET path = /timesync sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8282025-04-30T06:14:03.551ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = DELETE path = /v2p sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8292025-04-30T06:14:03.551ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = GET path = /v2p sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8302025-04-30T06:14:03.551ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = PUT path = /v2p sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8312025-04-30T06:14:03.551ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = DELETE path = /vmms/{propolis_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8322025-04-30T06:14:03.551ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = PUT path = /vmms/{propolis_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8332025-04-30T06:14:03.551ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = POST path = /vmms/{propolis_id}/disks/{disk_id}/snapshot sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8342025-04-30T06:14:03.551ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = DELETE path = /vmms/{propolis_id}/external-ip sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8352025-04-30T06:14:03.551ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = PUT path = /vmms/{propolis_id}/external-ip sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8362025-04-30T06:14:03.551ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = POST path = /vmms/{propolis_id}/poke sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8372025-04-30T06:14:03.551ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = POST path = /vmms/{propolis_id}/poke-single-step sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8382025-04-30T06:14:03.551ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = POST path = /vmms/{propolis_id}/sim-migration-source sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8392025-04-30T06:14:03.552ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = GET path = /vmms/{propolis_id}/state sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8402025-04-30T06:14:03.552ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = PUT path = /vmms/{propolis_id}/state sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8412025-04-30T06:14:03.552ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = PUT path = /vpc/{vpc_id}/firewall/rules sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8422025-04-30T06:14:03.552ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = GET path = /vpc-routes sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8432025-04-30T06:14:03.552ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = PUT path = /vpc-routes sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8442025-04-30T06:14:03.552ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = GET path = /zones sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8452025-04-30T06:14:03.552ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = POST path = /zones/bundle-cleanup sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8462025-04-30T06:14:03.552ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = GET path = /zones/bundle-cleanup/context sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8472025-04-30T06:14:03.552ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = PUT path = /zones/bundle-cleanup/context sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8482025-04-30T06:14:03.552ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = GET path = /zones/bundle-cleanup/utilization sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8492025-04-30T06:14:03.552ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = GET path = /zones/bundles sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8502025-04-30T06:14:03.552ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = GET path = /zones/bundles/{zone_name} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8512025-04-30T06:14:03.552ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = DELETE path = /zones/bundles/{zone_name}/{bundle_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8522025-04-30T06:14:03.552ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = GET path = /zones/bundles/{zone_name}/{bundle_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8532025-04-30T06:14:03.552ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:42420 method = GET path = /zpools sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8542025-04-30T06:14:03.552ZINFOtest_basic (dropshot): listening local_addr = [::1]:42420 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
8552025-04-30T06:14:03.552ZDEBGtest_basic (dropshot): successfully registered DTrace USDT probes local_addr = [::1]:42420 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
8562025-04-30T06:14:03.552ZDEBGtest_basic (omicron_sled_agent::sim::Server): contacting server nexus sled_id = b6d65341-167c-41df-9b5c-41cded99c229
8572025-04-30T06:14:03.559ZDEBGtest_basic (NexusClient): client request body = Some(Body) method = POST sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = http://127.0.0.1:34211/sled-agents/b6d65341-167c-41df-9b5c-41cded99c229
8582025-04-30T06:14:03.560ZTRACtest_basic (dns): buffer buffer = Length: 75 (0x4b) bytes\n0000: cd 92 01 00 00 01 00 00 00 00 00 01 0c 5f 6e 61 ............._na\n0010: 6d 65 73 65 72 76 69 63 65 04 5f 74 63 70 0d 63 meservice._tcp.c\n0020: 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 ontrol-plane.oxi\n0030: 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 de.internal..!..\n0040: 00 00 29 04 d0 00 00 00 00 00 00 ..)........ kind = dns peer_addr = [::1]:59236 req_id = 973d47a4-8a36-4b67-a0d0-9e316cd12f4d
8592025-04-30T06:14:03.560ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 52626,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_nameservice._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_nameservice._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 12,\n 95,\n 110,\n 97,\n 109,\n 101,\n 115,\n 101,\n 114,\n 118,\n 105,\n 99,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:59236 req_id = 973d47a4-8a36-4b67-a0d0-9e316cd12f4d
8602025-04-30T06:14:03.561ZERROtest_basic (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 52626, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_nameservice._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 12, 95, 110, 97, 109, 101, 115, 101, 114, 118, 105, 99, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_nameservice._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:59236 req_id = 973d47a4-8a36-4b67-a0d0-9e316cd12f4d
8612025-04-30T06:14:03.561ZTRACtest_basic (dns): buffer buffer = Length: 76 (0x4c) bytes\n0000: fd 08 01 00 00 01 00 00 00 00 00 01 0d 5f 65 78 ............._ex\n0010: 74 65 72 6e 61 6c 2d 64 6e 73 04 5f 74 63 70 0d ternal-dns._tcp.\n0020: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0030: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 21 00 ide.internal..!.\n0040: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:65418 req_id = a38930a7-35d3-445c-8119-e7bb879443ca
8622025-04-30T06:14:03.561ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 64776,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_external-dns._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_external-dns._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 13,\n 95,\n 101,\n 120,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 45,\n 100,\n 110,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:65418 req_id = a38930a7-35d3-445c-8119-e7bb879443ca
8632025-04-30T06:14:03.562ZERROtest_basic (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 64776, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_external-dns._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 13, 95, 101, 120, 116, 101, 114, 110, 97, 108, 45, 100, 110, 115, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_external-dns._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:65418 req_id = a38930a7-35d3-445c-8119-e7bb879443ca
8642025-04-30T06:14:03.562ZTRACtest_basic (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig
8652025-04-30T06:14:03.562ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_internal dns_group = internal resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
8662025-04-30T06:14:03.562ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
8672025-04-30T06:14:03.562ZDEBGtest_basic (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
8682025-04-30T06:14:03.563ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
8692025-04-30T06:14:03.564ZINFOtest_basic (dropshot_internal): accepted connection local_addr = 127.0.0.1:34211 remote_addr = 127.0.0.1:37560
8702025-04-30T06:14:03.564ZWARNtest_basic (BackgroundTasks): failed to lookup DNS servers background_task = dns_servers_internal dns_group = internal error = no record found for Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
8712025-04-30T06:14:03.564ZDEBGtest_basic (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 129.657314ms iteration = 1 status = {"error":"failed to read list of DNS servers: no record found for Query { name: Name(\\"_nameservice._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }"}
8722025-04-30T06:14:03.565ZWARNtest_basic (BackgroundTasks): failed to lookup DNS servers background_task = dns_servers_external dns_group = external error = no record found for Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
8732025-04-30T06:14:03.565ZDEBGtest_basic (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 128.898873ms iteration = 1 status = {"error":"failed to read list of DNS servers: no record found for Query { name: Name(\\"_external-dns._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }"}
8742025-04-30T06:14:03.565ZTRACtest_basic (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig
8752025-04-30T06:14:03.565ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_external dns_group = external resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
8762025-04-30T06:14:03.566ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
8772025-04-30T06:14:03.566ZDEBGtest_basic (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
8782025-04-30T06:14:03.566ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
8792025-04-30T06:14:03.567ZTRACtest_basic (dropshot_internal): incoming request local_addr = 127.0.0.1:34211 method = POST remote_addr = 127.0.0.1:37560 req_id = 457624d2-fdeb-4916-b4c8-ba4a8db3ab65 uri = /sled-agents/b6d65341-167c-41df-9b5c-41cded99c229
8802025-04-30T06:14:03.569ZINFOtest_basic (nexus): registered sled agent sled_uuid = b6d65341-167c-41df-9b5c-41cded99c229
8812025-04-30T06:14:03.579ZINFOtest_basic (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-04-30 06:04:03.436507939 UTC pruned = {}
8822025-04-30T06:14:03.580ZDEBGtest_basic (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 143.722829ms iteration = 1 status = {"expiration":"2025-04-30T06:04:03.436507939Z","pruned":[]}
8832025-04-30T06:14:03.581ZTRACtest_basic (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
8842025-04-30T06:14:03.584ZTRACtest_basic (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 7a 42 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 zB..........._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:59208 req_id = e9858f14-6c11-442b-971d-0ead47d7f6c9
8852025-04-30T06:14:03.584ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 31298,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:59208 req_id = e9858f14-6c11-442b-971d-0ead47d7f6c9
8862025-04-30T06:14:03.584ZERROtest_basic (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 31298, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_dendrite._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 9, 95, 100, 101, 110, 100, 114, 105, 116, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_dendrite._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:59208 req_id = e9858f14-6c11-442b-971d-0ead47d7f6c9
8872025-04-30T06:14:03.584ZERROtest_basic (BackgroundTasks): failed to resolve addresses for Dendrite services background_task = nat_v4_garbage_collector error = no record found for Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
8882025-04-30T06:14:03.584ZERROtest_basic (BackgroundTasks): failed to resolve addresses for Dendrite services background_task = nat_v4_garbage_collector error = no record found for Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
8892025-04-30T06:14:03.584ZDEBGtest_basic (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 146.575783ms iteration = 1 status = {"error":"failed to resolve addresses for Dendrite services: no record found for Query { name: Name(\\"_dendrite._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }"}
8902025-04-30T06:14:03.603ZTRACtest_basic (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
8912025-04-30T06:14:03.611ZDEBGtest_basic (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 172.076391ms iteration = 1 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
8922025-04-30T06:14:03.613ZTRACtest_basic (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 0c e6 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 ............._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:54692 req_id = 0f1ae779-fe7c-4032-8e9f-290f3a4ad48a
8932025-04-30T06:14:03.615ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 3302,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54692 req_id = 0f1ae779-fe7c-4032-8e9f-290f3a4ad48a
8942025-04-30T06:14:03.617ZERROtest_basic (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 3302, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_dendrite._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 9, 95, 100, 101, 110, 100, 114, 105, 116, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_dendrite._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:54692 req_id = 0f1ae779-fe7c-4032-8e9f-290f3a4ad48a
8952025-04-30T06:14:03.618ZDEBGtest_basic (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8962025-04-30T06:14:03.636ZDEBGtest_basic (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
8972025-04-30T06:14:03.636ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
8982025-04-30T06:14:03.636ZDEBGtest_basic (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
8992025-04-30T06:14:03.637ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
9002025-04-30T06:14:03.639ZDEBGtest_basic (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9012025-04-30T06:14:03.667ZDEBGtest_basic (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
9022025-04-30T06:14:03.668ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
9032025-04-30T06:14:03.669ZDEBGtest_basic (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
9042025-04-30T06:14:03.669ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
9052025-04-30T06:14:03.676ZDEBGtest_basic (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9062025-04-30T06:14:03.686ZDEBGtest_basic (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9072025-04-30T06:14:03.687ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database
9082025-04-30T06:14:03.687ZDEBGtest_basic (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9092025-04-30T06:14:03.688ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9102025-04-30T06:14:03.690ZERROtest_basic (BackgroundTasks): failed to resolve addresses for Dendrite services background_task = bfd_manager error = no record found for Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
9112025-04-30T06:14:03.690ZERROtest_basic (BackgroundTasks): failed to resolve addresses for Dendrite services background_task = bfd_manager error = no record found for Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
9122025-04-30T06:14:03.690ZDEBGtest_basic (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 252.045154ms iteration = 1 status = {"error":"failed to resolve addresses for Dendrite services: no record found for Query { name: Name(\\"_dendrite._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }"}
9132025-04-30T06:14:03.706ZDEBGtest_basic (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9142025-04-30T06:14:03.711ZDEBGtest_basic (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
9152025-04-30T06:14:03.711ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database
9162025-04-30T06:14:03.711ZDEBGtest_basic (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
9172025-04-30T06:14:03.711ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
9182025-04-30T06:14:03.717ZDEBGtest_basic (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9192025-04-30T06:14:03.749ZDEBGtest_basic (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
9202025-04-30T06:14:03.749ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database
9212025-04-30T06:14:03.749ZDEBGtest_basic (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
9222025-04-30T06:14:03.750ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
9232025-04-30T06:14:03.750ZDEBGtest_basic (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9242025-04-30T06:14:03.761ZDEBGtest_basic (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
9252025-04-30T06:14:03.761ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database
9262025-04-30T06:14:03.761ZDEBGtest_basic (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9272025-04-30T06:14:03.762ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9282025-04-30T06:14:03.794ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
9292025-04-30T06:14:03.827ZDEBGtest_basic (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
9302025-04-30T06:14:03.828ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
9312025-04-30T06:14:03.832ZDEBGtest_basic (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9322025-04-30T06:14:03.850ZDEBGtest_basic (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
9332025-04-30T06:14:03.850ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database
9342025-04-30T06:14:03.851ZDEBGtest_basic (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9352025-04-30T06:14:03.852ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9362025-04-30T06:14:03.865ZDEBGtest_basic (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9372025-04-30T06:14:03.886ZDEBGtest_basic (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
9382025-04-30T06:14:03.886ZTRACtest_basic (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet
9392025-04-30T06:14:03.886ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = vpc_route_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
9402025-04-30T06:14:03.886ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
9412025-04-30T06:14:03.886ZDEBGtest_basic (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9422025-04-30T06:14:03.886ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9432025-04-30T06:14:03.888ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database
9442025-04-30T06:14:03.888ZDEBGtest_basic (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9452025-04-30T06:14:03.889ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9462025-04-30T06:14:03.889ZDEBGtest_basic (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 424.41036ms iteration = 1 status = {"errors":[],"found_port_ok":[]}
9472025-04-30T06:14:03.890ZTRACtest_basic (BackgroundTasks): no more instances in need of reincarnation background_task = instance_reincarnation reincarnation_reason = instance failed total_found = 0
9482025-04-30T06:14:03.891ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database
9492025-04-30T06:14:03.891ZDEBGtest_basic (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
9502025-04-30T06:14:03.892ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
9512025-04-30T06:14:03.899ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database
9522025-04-30T06:14:03.899ZDEBGtest_basic (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
9532025-04-30T06:14:03.900ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
9542025-04-30T06:14:04.100ZINFOtest_basic (SagaRecoverer): listed in-progress sagas count = 0
9552025-04-30T06:14:04.109ZDEBGtest_basic (BackgroundTasks): activation complete background_task = saga_recovery elapsed = 644.891665ms iteration = 1 status = {"last_pass":{"Success":{"nfailed":0,"nfound":0,"nrecovered":0,"nremoved":0,"nskipped":0}},"ntotal_failures":0,"ntotal_finished":0,"ntotal_recovered":0,"ntotal_sec_errors_bad_state":0,"ntotal_sec_errors_missing":0,"ntotal_started":0,"recent_failures":[],"recent_recoveries":[]}
9562025-04-30T06:14:04.109ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
9572025-04-30T06:14:04.109ZDEBGtest_basic (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
9582025-04-30T06:14:04.111ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
9592025-04-30T06:14:04.141ZDEBGtest_basic (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
9602025-04-30T06:14:04.141ZINFOtest_basic (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
9612025-04-30T06:14:04.141ZDEBGtest_basic (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 678.281201ms iteration = 1 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
9622025-04-30T06:14:04.142ZDEBGtest_basic (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9632025-04-30T06:14:04.149ZDEBGtest_basic (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
9642025-04-30T06:14:04.149ZTRACtest_basic (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet
9652025-04-30T06:14:04.149ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = tuf_artifact_replication resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
9662025-04-30T06:14:04.149ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database
9672025-04-30T06:14:04.149ZDEBGtest_basic (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
9682025-04-30T06:14:04.150ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
9692025-04-30T06:14:04.166ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database
9702025-04-30T06:14:04.166ZDEBGtest_basic (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
9712025-04-30T06:14:04.167ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
9722025-04-30T06:14:04.178ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database
9732025-04-30T06:14:04.178ZDEBGtest_basic (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
9742025-04-30T06:14:04.179ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
9752025-04-30T06:14:04.180ZDEBGtest_basic (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 712.753014ms iteration = 1 status = {"errors":[],"garbage_collect_requested":[]}
9762025-04-30T06:14:04.180ZDEBGtest_basic (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 710.864181ms iteration = 1 status = {"errors":[],"finish_invoked_ok":[]}
9772025-04-30T06:14:04.194ZDEBGtest_basic (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 716.260893ms iteration = 1 status = {"by_rx":{},"error":null}
9782025-04-30T06:14:04.202ZDEBGtest_basic (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9792025-04-30T06:14:04.232ZDEBGtest_basic (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
9802025-04-30T06:14:04.236ZDEBGtest_basic (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 759.092503ms iteration = 1 status = {"errors":[],"requests_created_ok":[]}
9812025-04-30T06:14:04.262ZDEBGtest_basic (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9822025-04-30T06:14:04.273ZDEBGtest_basic (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
9832025-04-30T06:14:04.285ZTRACtest_basic (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig
9842025-04-30T06:14:04.285ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = external_endpoints resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
9852025-04-30T06:14:04.286ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
9862025-04-30T06:14:04.287ZDEBGtest_basic (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
9872025-04-30T06:14:04.289ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
9882025-04-30T06:14:04.291ZDEBGtest_basic (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = []
9892025-04-30T06:14:04.291ZTRACtest_basic (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
9902025-04-30T06:14:04.293ZTRACtest_basic (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: c6 75 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .u..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:65256 req_id = c0f4339d-b585-41cf-8249-694eb3d3cf6e
9912025-04-30T06:14:04.294ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 50805,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:65256 req_id = c0f4339d-b585-41cf-8249-694eb3d3cf6e
9922025-04-30T06:14:04.299ZERROtest_basic (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 50805, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_mgs._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 4, 95, 109, 103, 115, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_mgs._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:65256 req_id = c0f4339d-b585-41cf-8249-694eb3d3cf6e
9932025-04-30T06:14:04.300ZWARNtest_basic (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: looking up MGS addresses: no record found for Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
9942025-04-30T06:14:04.301ZDEBGtest_basic (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 861.344001ms iteration = 1 status = {"error":"failed to collect inventory: looking up MGS addresses: no record found for Query { name: Name(\\"_mgs._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }"}
9952025-04-30T06:14:04.316ZWARNtest_basic (BackgroundTasks): failed to read DNS config background_task = dns_config_external dns_group = external error = Internal Error: expected exactly one latest version for DNS group External, found 0
9962025-04-30T06:14:04.319ZDEBGtest_basic (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 883.922879ms iteration = 1 status = {"error":"failed to read DNS config: Internal Error: expected exactly one latest version for DNS group External, found 0"}
9972025-04-30T06:14:04.320ZWARNtest_basic (BackgroundTasks): failed to read target blueprint background_task = blueprint_loader error = Internal Error: no target blueprint set
9982025-04-30T06:14:04.320ZDEBGtest_basic (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 881.617496ms iteration = 1 status = {"error":"failed to read target blueprint: Internal Error: no target blueprint set"}
9992025-04-30T06:14:04.328ZWARNtest_basic (BackgroundTasks): inventory collection is None background_task = service_zone_nat_tracker
10002025-04-30T06:14:04.329ZDEBGtest_basic (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 887.823861ms iteration = 1 status = {"error":"inventory collection is None"}
10012025-04-30T06:14:04.331ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
10022025-04-30T06:14:04.331ZDEBGtest_basic (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
10032025-04-30T06:14:04.333ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
10042025-04-30T06:14:04.351ZDEBGtest_basic (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 908.577231ms iteration = 1 status = {}
10052025-04-30T06:14:04.354ZDEBGtest_basic (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10062025-04-30T06:14:04.359ZDEBGtest_basic (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
10072025-04-30T06:14:04.359ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
10082025-04-30T06:14:04.359ZDEBGtest_basic (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10092025-04-30T06:14:04.360ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10102025-04-30T06:14:04.395ZTRACtest_basic (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet
10112025-04-30T06:14:04.395ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = support_bundle_collector resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
10122025-04-30T06:14:04.395ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database
10132025-04-30T06:14:04.395ZDEBGtest_basic (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
10142025-04-30T06:14:04.395ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
10152025-04-30T06:14:04.396ZWARNtest_basic (BackgroundTasks): failed to read DNS config background_task = dns_config_internal dns_group = internal error = Internal Error: expected exactly one latest version for DNS group Internal, found 0
10162025-04-30T06:14:04.396ZDEBGtest_basic (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 962.320511ms iteration = 1 status = {"error":"failed to read DNS config: Internal Error: expected exactly one latest version for DNS group Internal, found 0"}
10172025-04-30T06:14:04.398ZDEBGtest_basic (BackgroundTasks): activation complete background_task = region_replacement elapsed = 950.983453ms iteration = 1 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
10182025-04-30T06:14:04.500ZDEBGtest_basic (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10192025-04-30T06:14:04.531ZDEBGtest_basic (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
10202025-04-30T06:14:04.531ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
10212025-04-30T06:14:04.531ZDEBGtest_basic (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
10222025-04-30T06:14:04.533ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
10232025-04-30T06:14:04.533ZTRACtest_basic (BackgroundTasks): reprocessing 0 outdated webhook globs... background_task = webhook_dispatcher
10242025-04-30T06:14:04.534ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_dispatcher resource = Database
10252025-04-30T06:14:04.534ZDEBGtest_basic (BackgroundTasks): roles background_task = webhook_dispatcher roles = RoleSet { roles: {} }
10262025-04-30T06:14:04.534ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_dispatcher resource = Database result = Ok(())
10272025-04-30T06:14:04.547ZDEBGtest_basic (BackgroundTasks): activating background_task = inventory_collection iteration = 2 reason = Signaled
10282025-04-30T06:14:04.547ZTRACtest_basic (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
10292025-04-30T06:14:04.548ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
10302025-04-30T06:14:04.548ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
10312025-04-30T06:14:04.548ZDEBGtest_basic (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
10322025-04-30T06:14:04.549ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
10332025-04-30T06:14:04.549ZINFOtest_basic (dropshot_internal): request completed latency_us = 981487 local_addr = 127.0.0.1:34211 method = POST remote_addr = 127.0.0.1:37560 req_id = 457624d2-fdeb-4916-b4c8-ba4a8db3ab65 response_code = 204 uri = /sled-agents/b6d65341-167c-41df-9b5c-41cded99c229
10342025-04-30T06:14:04.550ZDEBGtest_basic (NexusClient): client response result = Ok(Response { url: "http://127.0.0.1:34211/sled-agents/b6d65341-167c-41df-9b5c-41cded99c229", status: 204, headers: {"x-request-id": "457624d2-fdeb-4916-b4c8-ba4a8db3ab65", "date": "Wed, 30 Apr 2025 06:14:03 GMT"} }) sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10352025-04-30T06:14:04.550ZDEBGtest_basic (init_with_steps): Running step start_crucible_pantry
10362025-04-30T06:14:04.552ZDEBGtest_basic (dropshot): registered endpoint kind = pantry local_addr = [::1]:43262 method = GET path = /crucible/pantry/0 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10372025-04-30T06:14:04.552ZDEBGtest_basic (dropshot): registered endpoint kind = pantry local_addr = [::1]:43262 method = GET path = /crucible/pantry/0/job/{id}/is-finished sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10382025-04-30T06:14:04.552ZDEBGtest_basic (dropshot): registered endpoint kind = pantry local_addr = [::1]:43262 method = GET path = /crucible/pantry/0/job/{id}/ok sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10392025-04-30T06:14:04.552ZDEBGtest_basic (dropshot): registered endpoint kind = pantry local_addr = [::1]:43262 method = DELETE path = /crucible/pantry/0/volume/{id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10402025-04-30T06:14:04.552ZDEBGtest_basic (dropshot): registered endpoint kind = pantry local_addr = [::1]:43262 method = GET path = /crucible/pantry/0/volume/{id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10412025-04-30T06:14:04.552ZDEBGtest_basic (dropshot): registered endpoint kind = pantry local_addr = [::1]:43262 method = POST path = /crucible/pantry/0/volume/{id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10422025-04-30T06:14:04.552ZDEBGtest_basic (dropshot): registered endpoint kind = pantry local_addr = [::1]:43262 method = POST path = /crucible/pantry/0/volume/{id}/background sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10432025-04-30T06:14:04.552ZDEBGtest_basic (dropshot): registered endpoint kind = pantry local_addr = [::1]:43262 method = POST path = /crucible/pantry/0/volume/{id}/bulk-write sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10442025-04-30T06:14:04.552ZDEBGtest_basic (dropshot): registered endpoint kind = pantry local_addr = [::1]:43262 method = POST path = /crucible/pantry/0/volume/{id}/import-from-url sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10452025-04-30T06:14:04.552ZDEBGtest_basic (dropshot): registered endpoint kind = pantry local_addr = [::1]:43262 method = POST path = /crucible/pantry/0/volume/{id}/scrub sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10462025-04-30T06:14:04.552ZDEBGtest_basic (dropshot): registered endpoint kind = pantry local_addr = [::1]:43262 method = POST path = /crucible/pantry/0/volume/{id}/snapshot sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10472025-04-30T06:14:04.552ZINFOtest_basic (dropshot): listening kind = pantry local_addr = [::1]:43262 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10482025-04-30T06:14:04.553ZDEBGtest_basic (dropshot): successfully registered DTrace USDT probes kind = pantry local_addr = [::1]:43262 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10492025-04-30T06:14:04.553ZINFOtest_basic (omicron_sled_agent::sim::Server): Started Simulated Crucible Pantry address = [::1]:43262 kind = pantry sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10502025-04-30T06:14:04.553ZDEBGtest_basic (init_with_steps): Running step populate_internal_dns
10512025-04-30T06:14:04.553ZDEBGtest_basic: Populating Internal DNS
10522025-04-30T06:14:04.640ZINFOtest_basic: DNS population: DnsConfigParams { generation: Generation( 1, ), time_created: 2025-04-30T06:14:04.640141029Z, zones: [ DnsConfigZone { zone_name: "control-plane.oxide.internal", records: { "_nameservice._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 57818, target: "bab7bfc0-bde9-4584-a405-df241034d9fc.host.control-plane.oxide.internal", }, ), ], "b6d65341-167c-41df-9b5c-41cded99c229.sled": [ Aaaa( ::1, ), ], "_oximeter-reader._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 9000, target: "40d5053c-8b54-4f5d-ba94-d08c3988e280.host.control-plane.oxide.internal", }, ), ], "40d5053c-8b54-4f5d-ba94-d08c3988e280.host": [ Aaaa( ::1, ), ], "6880dc28-d971-4307-bf8a-dd2d981a337c.host": [ Aaaa( ::1, ), ], "_external-dns._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 46146, target: "c4e9f997-f271-4a58-92bc-01625e84ce01.host.control-plane.oxide.internal", }, ), ], "_clickhouse-admin-single-server._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 8888, target: "40d5053c-8b54-4f5d-ba94-d08c3988e280.host.control-plane.oxide.internal", }, ), ], "_nexus._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 34211, target: "088ac96c-981b-4054-bea6-d8ab23ae1366.host.control-plane.oxide.internal", }, ), ], "3c2b6b58-ebb9-4a1f-8d6f-d17b017d44cb.host": [ Aaaa( ::1, ), ], "c4e9f997-f271-4a58-92bc-01625e84ce01.host": [ Aaaa( ::1, ), ], "_cockroach._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 47002, target: "3c2b6b58-ebb9-4a1f-8d6f-d17b017d44cb.host.control-plane.oxide.internal", }, ), ], "_dendrite._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 55252, target: "dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal", }, ), ], "088ac96c-981b-4054-bea6-d8ab23ae1366.host": [ Aaaa( ::ffff:127.0.0.1, ), ], "_clickhouse-native._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 9000, target: "40d5053c-8b54-4f5d-ba94-d08c3988e280.host.control-plane.oxide.internal", }, ), ], "_mgd._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 51711, target: "dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal", }, ), ], "bab7bfc0-bde9-4584-a405-df241034d9fc.host": [ Aaaa( ::1, ), ], "_repo-depot._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 51632, target: "b6d65341-167c-41df-9b5c-41cded99c229.sled.control-plane.oxide.internal", }, ), ], "_mgs._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 37822, target: "dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal", }, ), ], "_crucible-pantry._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 43262, target: "6880dc28-d971-4307-bf8a-dd2d981a337c.host.control-plane.oxide.internal", }, ), ], "_clickhouse._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 46986, target: "40d5053c-8b54-4f5d-ba94-d08c3988e280.host.control-plane.oxide.internal", }, ), ], "dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host": [ Aaaa( ::1, ), ], }, }, ], }
10532025-04-30T06:14:04.640ZDEBGtest_basic: client request body = Some(Body) method = PUT uri = http://[::1]:57818/config
10542025-04-30T06:14:04.641ZINFOtest_basic (BackgroundTasks): all instance checks complete background_task = instance_watcher pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
10552025-04-30T06:14:04.641ZDEBGtest_basic (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 1.191907217s iteration = 1 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
10562025-04-30T06:14:04.656ZDEBGtest_basic (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 1.208579607s iteration = 1 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
10572025-04-30T06:14:04.657ZTRACtest_basic (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
10582025-04-30T06:14:04.657ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
10592025-04-30T06:14:04.657ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
10602025-04-30T06:14:04.658ZDEBGtest_basic (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
10612025-04-30T06:14:04.658ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
10622025-04-30T06:14:04.659ZDEBGtest_basic (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 1.192780237s iteration = 1 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
10632025-04-30T06:14:04.659ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database
10642025-04-30T06:14:04.659ZDEBGtest_basic (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
10652025-04-30T06:14:04.659ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
10662025-04-30T06:14:04.660ZDEBGtest_basic (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10672025-04-30T06:14:04.664ZDEBGtest_basic (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
10682025-04-30T06:14:04.664ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database
10692025-04-30T06:14:04.664ZDEBGtest_basic (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
10702025-04-30T06:14:04.665ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
10712025-04-30T06:14:04.666ZDEBGtest_basic (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10722025-04-30T06:14:04.722ZDEBGtest_basic (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
10732025-04-30T06:14:04.723ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database
10742025-04-30T06:14:04.724ZDEBGtest_basic (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
10752025-04-30T06:14:04.726ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
10762025-04-30T06:14:04.729ZINFOtest_basic (http): accepted connection kind = dns local_addr = [::1]:57818 remote_addr = [::1]:63319
10772025-04-30T06:14:04.730ZTRACtest_basic (http): incoming request kind = dns local_addr = [::1]:57818 method = PUT remote_addr = [::1]:63319 req_id = 76e15452-121b-4945-87bd-6d7e92229b7c uri = /config
10782025-04-30T06:14:04.732ZINFOtest_basic (store): attempting generation update new_generation = 1 req_id = 76e15452-121b-4945-87bd-6d7e92229b7c
10792025-04-30T06:14:04.732ZINFOtest_basic (store): pruning trees for generations newer than 0
10802025-04-30T06:14:04.732ZDEBGtest_basic (store): creating tree tree_name = generation_1_zone_control-plane.oxide.internal
10812025-04-30T06:14:04.738ZDEBGtest_basic (store): updating current config
10822025-04-30T06:14:04.739ZDEBGtest_basic (store): flushing default tree
10832025-04-30T06:14:04.739ZINFOtest_basic (store): pruning trees for generations older than 1 keep = 3
10842025-04-30T06:14:04.739ZINFOtest_basic (store): updated generation new_generation = 1 req_id = 76e15452-121b-4945-87bd-6d7e92229b7c
10852025-04-30T06:14:04.740ZINFOtest_basic (http): request completed kind = dns latency_us = 9861 local_addr = [::1]:57818 method = PUT remote_addr = [::1]:63319 req_id = 76e15452-121b-4945-87bd-6d7e92229b7c response_code = 204 uri = /config
10862025-04-30T06:14:04.740ZDEBGtest_basic: client response result = Ok(Response { url: "http://[::1]:57818/config", status: 204, headers: {"x-request-id": "76e15452-121b-4945-87bd-6d7e92229b7c", "date": "Wed, 30 Apr 2025 06:14:04 GMT"} })
10872025-04-30T06:14:04.742ZDEBGtest_basic (init_with_steps): Running step configure_sled_agents
10882025-04-30T06:14:04.840ZDEBGtest_basic: client request body = Some(Body) method = PUT uri = http://[::1]:42420/omicron-config
10892025-04-30T06:14:04.841ZTRACtest_basic (BackgroundTasks): webhook dispatching completed successfully background_task = webhook_dispatcher events_dispatched = 0 events_without_receivers = 0
10902025-04-30T06:14:04.841ZDEBGtest_basic (BackgroundTasks): activation complete background_task = webhook_dispatcher elapsed = 1.36438136s iteration = 1 status = {"dispatched":[],"errors":[],"glob_version":"140.0.0","globs_reprocessed":{},"no_receivers":[]}
10912025-04-30T06:14:04.842ZWARNtest_basic (BackgroundTasks): failed to read Silo/DNS/TLS configuration background_task = external_endpoints error = Internal Error: expected at least one external DNS zone
10922025-04-30T06:14:04.842ZDEBGtest_basic (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 1.404929961s iteration = 1 status = {"error":"failed to read Silo/DNS/TLS configuration: Internal Error: expected at least one external DNS zone"}
10932025-04-30T06:14:04.842ZTRACtest_basic (BackgroundTasks): no more instances in need of reincarnation background_task = instance_reincarnation reincarnation_reason = start saga failed total_found = 0
10942025-04-30T06:14:04.842ZINFOtest_basic (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
10952025-04-30T06:14:04.842ZDEBGtest_basic (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 1.387294012s iteration = 1 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
10962025-04-30T06:14:04.842ZINFOtest_basic (dropshot): accepted connection local_addr = [::1]:42420 remote_addr = [::1]:56984 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10972025-04-30T06:14:04.843ZTRACtest_basic (dropshot): incoming request local_addr = [::1]:42420 method = PUT remote_addr = [::1]:56984 req_id = a51666aa-9320-47dc-a38a-db009fe5eae0 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /omicron-config
10982025-04-30T06:14:04.846ZINFOtest_basic (dropshot): request completed latency_us = 2464 local_addr = [::1]:42420 method = PUT remote_addr = [::1]:56984 req_id = a51666aa-9320-47dc-a38a-db009fe5eae0 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /omicron-config
10992025-04-30T06:14:04.846ZDEBGtest_basic: client response result = Ok(Response { url: "http://[::1]:42420/omicron-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "a51666aa-9320-47dc-a38a-db009fe5eae0", "content-length": "26", "date": "Wed, 30 Apr 2025 06:14:04 GMT"} })
11002025-04-30T06:14:04.846ZDEBGtest_basic: client request body = Some(Body) method = PUT uri = http://[::1]:42420/network-bootstore-config
11012025-04-30T06:14:04.847ZTRACtest_basic (dropshot): incoming request local_addr = [::1]:42420 method = PUT remote_addr = [::1]:56984 req_id = 13be1434-82cf-4def-98f7-71e57c48892c sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /network-bootstore-config
11022025-04-30T06:14:04.847ZINFOtest_basic (dropshot): request completed latency_us = 577 local_addr = [::1]:42420 method = PUT remote_addr = [::1]:56984 req_id = 13be1434-82cf-4def-98f7-71e57c48892c response_code = 204 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /network-bootstore-config
11032025-04-30T06:14:04.848ZDEBGtest_basic: client response result = Ok(Response { url: "http://[::1]:42420/network-bootstore-config", status: 204, headers: {"x-request-id": "13be1434-82cf-4def-98f7-71e57c48892c", "date": "Wed, 30 Apr 2025 06:14:04 GMT"} })
11042025-04-30T06:14:04.848ZDEBGtest_basic (init_with_steps): Running step start_nexus_external
11052025-04-30T06:14:04.849ZDEBGtest_basic: Starting Nexus (external API)
11062025-04-30T06:14:08.793ZTRACtest_basic (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
11072025-04-30T06:14:08.794ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
11082025-04-30T06:14:08.794ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
11092025-04-30T06:14:08.794ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
11102025-04-30T06:14:08.795ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
11112025-04-30T06:14:08.918ZINFOtest_basic (BackgroundTasks): VPC route manager sled b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager
11122025-04-30T06:14:08.918ZDEBGtest_basic (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager body = None method = GET uri = http://[::1]:42420/vpc-routes
11132025-04-30T06:14:08.919ZDEBGtest_basic (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11142025-04-30T06:14:08.926ZDEBGtest_basic (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
11152025-04-30T06:14:08.926ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
11162025-04-30T06:14:08.926ZDEBGtest_basic (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
11172025-04-30T06:14:08.928ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
11182025-04-30T06:14:08.928ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
11192025-04-30T06:14:08.928ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
11202025-04-30T06:14:08.928ZDEBGtest_basic (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
11212025-04-30T06:14:08.930ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
11222025-04-30T06:14:08.931ZTRACtest_basic (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet
11232025-04-30T06:14:08.931ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = v2p_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
11242025-04-30T06:14:08.931ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
11252025-04-30T06:14:08.931ZDEBGtest_basic (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
11262025-04-30T06:14:08.932ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
11272025-04-30T06:14:08.933ZINFOtest_basic (dropshot): accepted connection local_addr = [::1]:42420 remote_addr = [::1]:53645 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
11282025-04-30T06:14:08.939ZDEBGtest_basic (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 5.471170721s iteration = 1 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
11292025-04-30T06:14:08.942ZTRACtest_basic (dropshot): incoming request local_addr = [::1]:42420 method = GET remote_addr = [::1]:53645 req_id = 8c659ea9-b9a0-4584-a602-bf8ceea4fbd0 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc-routes
11302025-04-30T06:14:08.945ZINFOtest_basic (dropshot): request completed latency_us = 2414 local_addr = [::1]:42420 method = GET remote_addr = [::1]:53645 req_id = 8c659ea9-b9a0-4584-a602-bf8ceea4fbd0 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc-routes
11312025-04-30T06:14:08.949ZDEBGtest_basic (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager result = Ok(Response { url: "http://[::1]:42420/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "8c659ea9-b9a0-4584-a602-bf8ceea4fbd0", "content-length": "2", "date": "Wed, 30 Apr 2025 06:14:08 GMT"} })
11322025-04-30T06:14:08.958ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
11332025-04-30T06:14:08.958ZDEBGtest_basic (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11342025-04-30T06:14:08.960ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11352025-04-30T06:14:08.976ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11362025-04-30T06:14:08.984ZDEBGtest_basic (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
11372025-04-30T06:14:08.984ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
11382025-04-30T06:14:08.984ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
11392025-04-30T06:14:08.988ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
11402025-04-30T06:14:08.995ZDEBGtest_basic (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = []
11412025-04-30T06:14:08.996ZTRACtest_basic (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
11422025-04-30T06:14:09.006ZINFOtest_basic (BackgroundTasks): No bundles to collect background_task = support_bundle_collector
11432025-04-30T06:14:09.006ZDEBGtest_basic (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 5.566126264s iteration = 1 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
11442025-04-30T06:14:09.010ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
11452025-04-30T06:14:09.010ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
11462025-04-30T06:14:09.010ZDEBGtest_basic (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
11472025-04-30T06:14:09.011ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
11482025-04-30T06:14:09.013ZDEBGtest_basic (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11492025-04-30T06:14:09.032ZDEBGtest_basic (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
11502025-04-30T06:14:09.032ZTRACtest_basic (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet
11512025-04-30T06:14:09.032ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = v2p_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
11522025-04-30T06:14:09.032ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
11532025-04-30T06:14:09.032ZDEBGtest_basic (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
11542025-04-30T06:14:09.034ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
11552025-04-30T06:14:09.163ZTRACtest_basic (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet
11562025-04-30T06:14:09.163ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = tuf_artifact_replication resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
11572025-04-30T06:14:09.164ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database
11582025-04-30T06:14:09.164ZDEBGtest_basic (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
11592025-04-30T06:14:09.165ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
11602025-04-30T06:14:09.172ZTRACtest_basic (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 84 70 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .p..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:63342 req_id = d3e03d03-b507-4b1d-9a46-071809225406
11612025-04-30T06:14:09.172ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 33904,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63342 req_id = d3e03d03-b507-4b1d-9a46-071809225406
11622025-04-30T06:14:09.173ZDEBGtest_basic (store): query key key = _mgs._tcp
11632025-04-30T06:14:09.173ZDEBGtest_basic (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
11642025-04-30T06:14:09.173ZDEBGtest_basic (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:63342 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 37822, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = d3e03d03-b507-4b1d-9a46-071809225406
11652025-04-30T06:14:09.177ZDEBGtest_basic (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 37822, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2513, tv_nsec: 976013678 } })
11662025-04-30T06:14:09.177ZTRACtest_basic (DnsResolver): lookup_service_targets: looking up SRV target
11672025-04-30T06:14:09.183ZDEBGtest_basic (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11682025-04-30T06:14:09.197ZDEBGtest_basic (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
11692025-04-30T06:14:09.197ZTRACtest_basic (BackgroundTasks): successful lookup of siloed resource "Vpc" using built-in user background_task = service_firewall_rule_propagation
11702025-04-30T06:14:09.197ZTRACtest_basic (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
11712025-04-30T06:14:09.197ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
11722025-04-30T06:14:09.198ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
11732025-04-30T06:14:09.198ZDEBGtest_basic (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
11742025-04-30T06:14:09.199ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
11752025-04-30T06:14:09.201ZDEBGtest_basic (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11762025-04-30T06:14:09.209ZDEBGtest_basic (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
11772025-04-30T06:14:09.209ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
11782025-04-30T06:14:09.209ZDEBGtest_basic (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
11792025-04-30T06:14:09.210ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
11802025-04-30T06:14:09.212ZDEBGtest_basic (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11812025-04-30T06:14:09.226ZDEBGtest_basic (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
11822025-04-30T06:14:09.226ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database
11832025-04-30T06:14:09.226ZDEBGtest_basic (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
11842025-04-30T06:14:09.226ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
11852025-04-30T06:14:09.227ZINFOtest_basic (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager sled = sim-b6d65341
11862025-04-30T06:14:09.227ZDEBGtest_basic (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[::1]:42420/eip-gateways
11872025-04-30T06:14:09.229ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
11882025-04-30T06:14:09.229ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
11892025-04-30T06:14:09.229ZDEBGtest_basic (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
11902025-04-30T06:14:09.230ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
11912025-04-30T06:14:09.232ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
11922025-04-30T06:14:09.232ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
11932025-04-30T06:14:09.233ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
11942025-04-30T06:14:09.235ZTRACtest_basic (dropshot): incoming request local_addr = [::1]:42420 method = PUT remote_addr = [::1]:53645 req_id = f36f7139-6074-405a-a87c-c1f551901887 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /eip-gateways
11952025-04-30T06:14:09.235ZTRACtest_basic (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 40 a5 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e @...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:53425 req_id = b6cdb084-adec-4eca-8b37-24b6bb49140c
11962025-04-30T06:14:09.235ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 16549,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:53425 req_id = b6cdb084-adec-4eca-8b37-24b6bb49140c
11972025-04-30T06:14:09.235ZDEBGtest_basic (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
11982025-04-30T06:14:09.235ZDEBGtest_basic (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:53425 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = b6cdb084-adec-4eca-8b37-24b6bb49140c
11992025-04-30T06:14:09.237ZINFOtest_basic (dropshot): request completed latency_us = 2542 local_addr = [::1]:42420 method = PUT remote_addr = [::1]:53645 req_id = f36f7139-6074-405a-a87c-c1f551901887 response_code = 204 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /eip-gateways
12002025-04-30T06:14:09.238ZTRACtest_basic (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet
12012025-04-30T06:14:09.238ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = tuf_artifact_replication resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
12022025-04-30T06:14:09.238ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database
12032025-04-30T06:14:09.238ZDEBGtest_basic (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
12042025-04-30T06:14:09.239ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
12052025-04-30T06:14:09.239ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
12062025-04-30T06:14:09.239ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
12072025-04-30T06:14:09.239ZDEBGtest_basic (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12082025-04-30T06:14:09.240ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12092025-04-30T06:14:09.501ZTRACtest_basic (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
12102025-04-30T06:14:09.512ZDEBGtest_basic (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager result = Ok(Response { url: "http://[::1]:42420/eip-gateways", status: 204, headers: {"x-request-id": "f36f7139-6074-405a-a87c-c1f551901887", "date": "Wed, 30 Apr 2025 06:14:08 GMT"} })
12112025-04-30T06:14:09.516ZDEBGtest_basic (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 6.054783622s iteration = 1 status = {}
12122025-04-30T06:14:09.516ZDEBGtest_basic (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12132025-04-30T06:14:09.529ZDEBGtest_basic (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
12142025-04-30T06:14:09.529ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database
12152025-04-30T06:14:09.529ZDEBGtest_basic (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
12162025-04-30T06:14:09.531ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
12172025-04-30T06:14:09.608ZDEBGtest_basic (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = v2p_manager body = None method = GET uri = http://[::1]:42420/v2p
12182025-04-30T06:14:09.610ZDEBGtest_basic (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12192025-04-30T06:14:09.627ZDEBGtest_basic (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
12202025-04-30T06:14:09.627ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
12212025-04-30T06:14:09.627ZDEBGtest_basic (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12222025-04-30T06:14:09.629ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12232025-04-30T06:14:09.630ZDEBGtest_basic (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = tuf_artifact_replication body = Some(Body) method = PUT uri = http://[::1]:42420/artifacts-config
12242025-04-30T06:14:09.634ZINFOtest_basic (InternalApi): set cockroachdb setting setting = cluster.preserve_downgrade_option value = 22.1
12252025-04-30T06:14:09.634ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
12262025-04-30T06:14:09.634ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
12272025-04-30T06:14:09.636ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
12282025-04-30T06:14:09.638ZINFOtest_basic (dropshot): accepted connection local_addr = [::1]:42420 remote_addr = [::1]:33487 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
12292025-04-30T06:14:09.638ZINFOtest_basic (dropshot): accepted connection local_addr = [::1]:42420 remote_addr = [::1]:40929 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
12302025-04-30T06:14:09.643ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
12312025-04-30T06:14:09.643ZDEBGtest_basic (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12322025-04-30T06:14:09.647ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12332025-04-30T06:14:09.660ZTRACtest_basic (dropshot): incoming request local_addr = [::1]:42420 method = GET remote_addr = [::1]:33487 req_id = e121015c-4253-445a-9608-3332ef9a359c sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /v2p
12342025-04-30T06:14:09.661ZTRACtest_basic (dropshot): incoming request local_addr = [::1]:42420 method = PUT remote_addr = [::1]:40929 req_id = 48578f28-517d-43c4-a3b7-343b43132d7c sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /artifacts-config
12352025-04-30T06:14:09.669ZTRACtest_basic (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 7e 57 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ~W..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:58257 req_id = a7a178da-d854-4d1d-804a-511615515a28
12362025-04-30T06:14:09.670ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 32343,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:58257 req_id = a7a178da-d854-4d1d-804a-511615515a28
12372025-04-30T06:14:09.670ZDEBGtest_basic (store): query key key = _clickhouse-admin-keeper._tcp
12382025-04-30T06:14:09.670ZERROtest_basic (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 32343, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:58257 req_id = a7a178da-d854-4d1d-804a-511615515a28
12392025-04-30T06:14:09.674ZINFOtest_basic (dropshot): request completed latency_us = 13830 local_addr = [::1]:42420 method = GET remote_addr = [::1]:33487 req_id = e121015c-4253-445a-9608-3332ef9a359c response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /v2p
12402025-04-30T06:14:09.674ZINFOtest_basic (ArtifactStore): No ledger in /var/tmp/omicron_tmp/.tmppTSgOY/artifact-config.json server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
12412025-04-30T06:14:09.675ZDEBGtest_basic (ArtifactStore): Failed to read ledger: Not found in storage path = /var/tmp/omicron_tmp/.tmppTSgOY/artifact-config.json server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
12422025-04-30T06:14:09.675ZINFOtest_basic (ArtifactStore): No ledger in /var/tmp/omicron_tmp/.tmp7hnHjr/artifact-config.json server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
12432025-04-30T06:14:09.675ZDEBGtest_basic (ArtifactStore): Failed to read ledger: Not found in storage path = /var/tmp/omicron_tmp/.tmp7hnHjr/artifact-config.json server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
12442025-04-30T06:14:09.675ZINFOtest_basic (ArtifactStore): Writing ledger to /var/tmp/omicron_tmp/.tmppTSgOY/.artifact-config.json.tmp server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
12452025-04-30T06:14:09.676ZINFOtest_basic (ArtifactStore): Writing ledger to /var/tmp/omicron_tmp/.tmp7hnHjr/.artifact-config.json.tmp server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
12462025-04-30T06:14:09.676ZTRACtest_basic (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Rack { parent: Fleet, key: c19a698f-c6f9-4a17-ae30-20d711b8f7dc, lookup_type: ById(c19a698f-c6f9-4a17-ae30-20d711b8f7dc) }
12472025-04-30T06:14:09.677ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
12482025-04-30T06:14:09.677ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
12492025-04-30T06:14:09.677ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
12502025-04-30T06:14:09.678ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
12512025-04-30T06:14:09.682ZDEBGtest_basic (BackgroundTasks): begin collection background_task = inventory_collection
12522025-04-30T06:14:09.683ZDEBGtest_basic (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:37822
12532025-04-30T06:14:09.683ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/ignition
12542025-04-30T06:14:09.685ZDEBGtest_basic (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = v2p_manager result = Ok(Response { url: "http://[::1]:42420/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "e121015c-4253-445a-9608-3332ef9a359c", "content-length": "2", "date": "Wed, 30 Apr 2025 06:14:08 GMT"} })
12552025-04-30T06:14:09.685ZINFOtest_basic (BackgroundTasks): found opte v2p mappings background_task = v2p_manager interfaces = {} sled = sim-b6d65341
12562025-04-30T06:14:09.685ZINFOtest_basic (BackgroundTasks): v2p mappings to delete background_task = v2p_manager mappings = [] sled = sim-b6d65341
12572025-04-30T06:14:09.685ZINFOtest_basic (BackgroundTasks): v2p mappings to add background_task = v2p_manager mappings = [] sled = sim-b6d65341
12582025-04-30T06:14:09.687ZDEBGtest_basic (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 6.241898947s iteration = 1 status = {}
12592025-04-30T06:14:09.688ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
12602025-04-30T06:14:09.688ZDEBGtest_basic (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12612025-04-30T06:14:09.691ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12622025-04-30T06:14:09.692ZINFOtest_basic (ArtifactStore): Starting delete reconciler generation = 1 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
12632025-04-30T06:14:09.692ZINFOtest_basic (dropshot): request completed latency_us = 31142 local_addr = [::1]:42420 method = PUT remote_addr = [::1]:40929 req_id = 48578f28-517d-43c4-a3b7-343b43132d7c response_code = 204 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /artifacts-config
12642025-04-30T06:14:09.712ZDEBGtest_basic (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = tuf_artifact_replication result = Ok(Response { url: "http://[::1]:42420/artifacts-config", status: 204, headers: {"x-request-id": "48578f28-517d-43c4-a3b7-343b43132d7c", "date": "Wed, 30 Apr 2025 06:14:08 GMT"} })
12652025-04-30T06:14:09.712ZDEBGtest_basic (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = tuf_artifact_replication body = None method = GET uri = http://[::1]:42420/artifacts
12662025-04-30T06:14:09.713ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
12672025-04-30T06:14:09.713ZDEBGtest_basic (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12682025-04-30T06:14:09.714ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12692025-04-30T06:14:09.715ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12702025-04-30T06:14:09.732ZDEBGtest_basic (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Rack { parent: Fleet, key: c19a698f-c6f9-4a17-ae30-20d711b8f7dc, lookup_type: ById(c19a698f-c6f9-4a17-ae30-20d711b8f7dc) } result = Ok(())
12712025-04-30T06:14:09.732ZDEBGtest_basic (InternalApi): updating rack subnet for rack c19a698f-c6f9-4a17-ae30-20d711b8f7dc to Some( V6( Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56, }, ), )
12722025-04-30T06:14:09.732ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
12732025-04-30T06:14:09.732ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
12742025-04-30T06:14:09.733ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
12752025-04-30T06:14:09.736ZTRACtest_basic (dropshot): incoming request local_addr = [::1]:42420 method = GET remote_addr = [::1]:40929 req_id = 78e50d8b-12ea-4c9a-85e7-f5c745f72166 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /artifacts
12762025-04-30T06:14:09.737ZINFOtest_basic (dropshot): request completed latency_us = 711 local_addr = [::1]:42420 method = GET remote_addr = [::1]:40929 req_id = 78e50d8b-12ea-4c9a-85e7-f5c745f72166 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /artifacts
12772025-04-30T06:14:09.743ZTRACtest_basic (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
12782025-04-30T06:14:09.743ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
12792025-04-30T06:14:09.743ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
12802025-04-30T06:14:09.743ZDEBGtest_basic (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12812025-04-30T06:14:09.744ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12822025-04-30T06:14:09.745ZDEBGtest_basic (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = tuf_artifact_replication result = Ok(Response { url: "http://[::1]:42420/artifacts", status: 200, headers: {"content-type": "application/json", "x-request-id": "78e50d8b-12ea-4c9a-85e7-f5c745f72166", "content-length": "26", "date": "Wed, 30 Apr 2025 06:14:08 GMT"} })
12832025-04-30T06:14:09.745ZINFOtest_basic (BackgroundTasks): Successfully got artifact list background_task = tuf_artifact_replication sled = http://[::1]:42420
12842025-04-30T06:14:09.756ZDEBGtest_basic (BackgroundTasks): activation complete background_task = tuf_artifact_replication elapsed = 6.28666028s iteration = 1 status = {"generation":1,"last_run_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":1,"put_config_err":0,"put_config_ok":1,"put_err":0,"put_ok":0},"lifetime_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":1,"put_config_err":0,"put_config_ok":1,"put_err":0,"put_ok":0},"local_repos":0,"request_debug_ringbuf":[{"generation":1,"operation":"put_config","target_sled":"b6d65341-167c-41df-9b5c-41cded99c229","time":"2025-04-30T06:14:09.712670744Z"},{"operation":"list","target_sled":"b6d65341-167c-41df-9b5c-41cded99c229","time":"2025-04-30T06:14:09.745811956Z"}]}
12852025-04-30T06:14:09.760ZINFOtest_basic (InternalApi): Using static configuration for external switchports
12862025-04-30T06:14:09.760ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
12872025-04-30T06:14:09.760ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
12882025-04-30T06:14:09.761ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
12892025-04-30T06:14:09.763ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
12902025-04-30T06:14:09.763ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
12912025-04-30T06:14:09.763ZDEBGtest_basic (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12922025-04-30T06:14:09.765ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12932025-04-30T06:14:09.789ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
12942025-04-30T06:14:09.789ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
12952025-04-30T06:14:09.790ZDEBGtest_basic (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12962025-04-30T06:14:09.791ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12972025-04-30T06:14:09.792ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "49a774c2-2776-4040-8fa2-9d0f937cc823", "content-length": "835", "date": "Wed, 30 Apr 2025 06:14:08 GMT"} })
12982025-04-30T06:14:09.794ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0
12992025-04-30T06:14:09.806ZDEBGtest_basic (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13002025-04-30T06:14:09.816ZDEBGtest_basic (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
13012025-04-30T06:14:09.816ZTRACtest_basic (BackgroundTasks): successful lookup of siloed resource "Vpc" using built-in user background_task = service_firewall_rule_propagation
13022025-04-30T06:14:09.816ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13032025-04-30T06:14:09.817ZDEBGtest_basic (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13042025-04-30T06:14:09.818ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13052025-04-30T06:14:09.825ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13062025-04-30T06:14:09.825ZDEBGtest_basic (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13072025-04-30T06:14:09.827ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13082025-04-30T06:14:09.841ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13092025-04-30T06:14:09.841ZDEBGtest_basic (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13102025-04-30T06:14:09.843ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13112025-04-30T06:14:09.846ZINFOtest_basic (InternalApi): Recording Rack Network Configuration
13122025-04-30T06:14:09.846ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
13132025-04-30T06:14:09.846ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
13142025-04-30T06:14:09.848ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
13152025-04-30T06:14:09.852ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13162025-04-30T06:14:09.853ZDEBGtest_basic (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13172025-04-30T06:14:09.853ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13182025-04-30T06:14:09.857ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e5a72164-041b-4473-869d-4909793a626a", "content-length": "734", "date": "Wed, 30 Apr 2025 06:14:08 GMT"} })
13192025-04-30T06:14:09.858ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=0
13202025-04-30T06:14:09.869ZTRACtest_basic (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") }
13212025-04-30T06:14:09.869ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
13222025-04-30T06:14:09.869ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13232025-04-30T06:14:09.869ZDEBGtest_basic (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13242025-04-30T06:14:09.870ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13252025-04-30T06:14:09.877ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
13262025-04-30T06:14:09.877ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13272025-04-30T06:14:09.877ZDEBGtest_basic (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13282025-04-30T06:14:09.879ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13292025-04-30T06:14:09.885ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
13302025-04-30T06:14:09.885ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13312025-04-30T06:14:09.885ZDEBGtest_basic (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13322025-04-30T06:14:09.887ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13332025-04-30T06:14:09.923ZDEBGtest_basic (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13342025-04-30T06:14:09.942ZDEBGtest_basic (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
13352025-04-30T06:14:09.942ZTRACtest_basic (BackgroundTasks): successful lookup of siloed resource "VpcSubnet" using built-in user background_task = service_firewall_rule_propagation
13362025-04-30T06:14:09.942ZTRACtest_basic (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") }
13372025-04-30T06:14:09.942ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
13382025-04-30T06:14:09.942ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13392025-04-30T06:14:09.942ZDEBGtest_basic (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13402025-04-30T06:14:09.943ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13412025-04-30T06:14:09.953ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
13422025-04-30T06:14:09.953ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13432025-04-30T06:14:09.953ZDEBGtest_basic (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13442025-04-30T06:14:09.955ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13452025-04-30T06:14:09.958ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
13462025-04-30T06:14:09.958ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13472025-04-30T06:14:09.958ZDEBGtest_basic (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13482025-04-30T06:14:09.959ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13492025-04-30T06:14:09.969ZTRACtest_basic (InternalApi): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
13502025-04-30T06:14:09.969ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
13512025-04-30T06:14:09.969ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
13522025-04-30T06:14:09.969ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
13532025-04-30T06:14:09.970ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
13542025-04-30T06:14:09.971ZDEBGtest_basic (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13552025-04-30T06:14:10.013ZDEBGtest_basic (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
13562025-04-30T06:14:10.014ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13572025-04-30T06:14:10.014ZDEBGtest_basic (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13582025-04-30T06:14:10.033ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13592025-04-30T06:14:10.040ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13602025-04-30T06:14:10.049ZDEBGtest_basic (InternalApi): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
13612025-04-30T06:14:10.049ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
13622025-04-30T06:14:10.049ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
13632025-04-30T06:14:10.050ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
13642025-04-30T06:14:10.052ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13652025-04-30T06:14:10.052ZDEBGtest_basic (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13662025-04-30T06:14:10.053ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13672025-04-30T06:14:10.058ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13682025-04-30T06:14:10.058ZDEBGtest_basic (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13692025-04-30T06:14:10.060ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13702025-04-30T06:14:10.062ZTRACtest_basic (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = IpPool { parent: Fleet, key: aaeb44fc-d05a-4984-9ecb-cce6845a69a8, lookup_type: ByName("oxide-service-pool") }
13712025-04-30T06:14:10.062ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
13722025-04-30T06:14:10.062ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
13732025-04-30T06:14:10.062ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
13742025-04-30T06:14:10.063ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
13752025-04-30T06:14:10.066ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13762025-04-30T06:14:10.066ZDEBGtest_basic (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13772025-04-30T06:14:10.067ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13782025-04-30T06:14:10.072ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13792025-04-30T06:14:10.083ZDEBGtest_basic (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = IpPool { parent: Fleet, key: aaeb44fc-d05a-4984-9ecb-cce6845a69a8, lookup_type: ByName("oxide-service-pool") } result = Ok(())
13802025-04-30T06:14:10.083ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
13812025-04-30T06:14:10.083ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
13822025-04-30T06:14:10.084ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
13832025-04-30T06:14:10.085ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13842025-04-30T06:14:10.085ZDEBGtest_basic (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13852025-04-30T06:14:10.086ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13862025-04-30T06:14:10.101ZTRACtest_basic (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") }
13872025-04-30T06:14:10.109ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
13882025-04-30T06:14:10.109ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13892025-04-30T06:14:10.110ZDEBGtest_basic (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13902025-04-30T06:14:10.111ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13912025-04-30T06:14:10.111ZTRACtest_basic (InternalApi): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = IpPool { parent: Fleet, key: aaeb44fc-d05a-4984-9ecb-cce6845a69a8, lookup_type: ByName("oxide-service-pool") }
13922025-04-30T06:14:10.122ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
13932025-04-30T06:14:10.122ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
13942025-04-30T06:14:10.122ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
13952025-04-30T06:14:10.123ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
13962025-04-30T06:14:10.155ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
13972025-04-30T06:14:10.155ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13982025-04-30T06:14:10.156ZDEBGtest_basic (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13992025-04-30T06:14:10.157ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14002025-04-30T06:14:10.166ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14012025-04-30T06:14:10.180ZDEBGtest_basic (InternalApi): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = IpPool { parent: Fleet, key: aaeb44fc-d05a-4984-9ecb-cce6845a69a8, lookup_type: ByName("oxide-service-pool") } result = Ok(())
14022025-04-30T06:14:10.209ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
14032025-04-30T06:14:10.209ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14042025-04-30T06:14:10.209ZDEBGtest_basic (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14052025-04-30T06:14:10.210ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14062025-04-30T06:14:10.210ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7bdd5928-5a91-42ef-ba82-9e1d8341e5e3", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:10 GMT"} })
14072025-04-30T06:14:10.210ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=1
14082025-04-30T06:14:10.214ZTRACtest_basic (InternalApi): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = IpPool { parent: Fleet, key: aaeb44fc-d05a-4984-9ecb-cce6845a69a8, lookup_type: ByName("oxide-service-pool") }
14092025-04-30T06:14:10.214ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
14102025-04-30T06:14:10.214ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
14112025-04-30T06:14:10.214ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
14122025-04-30T06:14:10.215ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
14132025-04-30T06:14:10.231ZDEBGtest_basic (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14142025-04-30T06:14:10.264ZDEBGtest_basic (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
14152025-04-30T06:14:10.266ZTRACtest_basic (BackgroundTasks): successful lookup of siloed resource "VpcSubnet" using built-in user background_task = service_firewall_rule_propagation
14162025-04-30T06:14:10.267ZTRACtest_basic (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") }
14172025-04-30T06:14:10.267ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
14182025-04-30T06:14:10.267ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14192025-04-30T06:14:10.267ZDEBGtest_basic (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14202025-04-30T06:14:10.268ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14212025-04-30T06:14:10.272ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14222025-04-30T06:14:10.312ZDEBGtest_basic (InternalApi): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = IpPool { parent: Fleet, key: aaeb44fc-d05a-4984-9ecb-cce6845a69a8, lookup_type: ByName("oxide-service-pool") } result = Ok(())
14232025-04-30T06:14:10.379ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
14242025-04-30T06:14:10.379ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14252025-04-30T06:14:10.379ZDEBGtest_basic (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14262025-04-30T06:14:10.380ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14272025-04-30T06:14:10.381ZTRACtest_basic (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = BlueprintConfig
14282025-04-30T06:14:10.381ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
14292025-04-30T06:14:10.382ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
14302025-04-30T06:14:10.382ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
14312025-04-30T06:14:10.384ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
14322025-04-30T06:14:10.388ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
14332025-04-30T06:14:10.388ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14342025-04-30T06:14:10.388ZDEBGtest_basic (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14352025-04-30T06:14:10.389ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14362025-04-30T06:14:10.403ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14372025-04-30T06:14:10.414ZDEBGtest_basic (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = BlueprintConfig result = Ok(())
14382025-04-30T06:14:10.419ZDEBGtest_basic (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14392025-04-30T06:14:10.458ZDEBGtest_basic (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
14402025-04-30T06:14:10.458ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14412025-04-30T06:14:10.459ZDEBGtest_basic (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14422025-04-30T06:14:10.463ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14432025-04-30T06:14:10.506ZDEBGtest_basic (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
14442025-04-30T06:14:10.506ZTRACtest_basic (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet
14452025-04-30T06:14:10.506ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
14462025-04-30T06:14:10.506ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14472025-04-30T06:14:10.507ZDEBGtest_basic (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14482025-04-30T06:14:10.509ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14492025-04-30T06:14:10.515ZDEBGtest_basic (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14502025-04-30T06:14:10.586ZDEBGtest_basic (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
14512025-04-30T06:14:10.586ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14522025-04-30T06:14:10.586ZDEBGtest_basic (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14532025-04-30T06:14:10.587ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14542025-04-30T06:14:10.621ZTRACtest_basic (service-firewall-rule-progator): fetched allowlist from DB allowed = AllowList { id: 001de000-a110-4000-8000-000000000000, time_created: 2025-04-30T06:06:08.248500Z, time_modified: 2025-04-30T06:06:08.248500Z, allowed_ips: None } background_task = service_firewall_rule_propagation
14552025-04-30T06:14:10.621ZDEBGtest_basic (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
14562025-04-30T06:14:10.621ZDEBGtest_basic (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = []
14572025-04-30T06:14:10.621ZDEBGtest_basic (service-firewall-rule-progator): resolved 0 rules for sleds background_task = service_firewall_rule_propagation
14582025-04-30T06:14:10.668ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "48b3dd40-ab43-4dfa-a3d2-bf612abae640", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:10 GMT"} })
14592025-04-30T06:14:10.668ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=0
14602025-04-30T06:14:10.718ZDEBGtest_basic (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = []
14612025-04-30T06:14:10.718ZDEBGtest_basic (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
14622025-04-30T06:14:10.718ZINFOtest_basic (service-firewall-rule-progator): updated firewall rules on 0 sleds background_task = service_firewall_rule_propagation
14632025-04-30T06:14:10.718ZDEBGtest_basic (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 7.259868824s
14642025-04-30T06:14:10.718ZDEBGtest_basic (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 7.260300344s iteration = 1 status = {}
14652025-04-30T06:14:10.720ZINFOtest_basic (InternalApi): inserted blueprint blueprint_id = 34c994cb-7b18-4649-a03a-2c57afcd0185
14662025-04-30T06:14:10.720ZTRACtest_basic (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = BlueprintConfig
14672025-04-30T06:14:10.720ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
14682025-04-30T06:14:10.721ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
14692025-04-30T06:14:10.721ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
14702025-04-30T06:14:10.725ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
14712025-04-30T06:14:10.732ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b975c14f-90cf-4c09-aed2-0bc6f182061b", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:10 GMT"} })
14722025-04-30T06:14:10.732ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=1
14732025-04-30T06:14:10.748ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14742025-04-30T06:14:10.757ZDEBGtest_basic (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = BlueprintConfig result = Ok(())
14752025-04-30T06:14:10.802ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "71f3ce6a-cae7-4241-950b-15b9aad52a07", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:10 GMT"} })
14762025-04-30T06:14:10.803ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=0
14772025-04-30T06:14:10.843ZINFOtest_basic (InternalApi): Inserted networking records for nexus service
14782025-04-30T06:14:10.843ZINFOtest_basic (InternalApi): No networking records needed for cockroach_db service
14792025-04-30T06:14:10.843ZINFOtest_basic (InternalApi): No networking records needed for clickhouse service
14802025-04-30T06:14:10.843ZINFOtest_basic (InternalApi): No networking records needed for crucible_pantry service
14812025-04-30T06:14:10.843ZINFOtest_basic (InternalApi): No networking records needed for internal_dns service
14822025-04-30T06:14:10.845ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3991fad8-5dae-4baa-ae30-cdad5322d9ed", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:10 GMT"} })
14832025-04-30T06:14:10.845ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=1
14842025-04-30T06:14:10.848ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "03142d05-5ff2-412b-bb83-0297b5283176", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:10 GMT"} })
14852025-04-30T06:14:10.848ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cmpa
14862025-04-30T06:14:10.849ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d6369724-b1a4-423a-a0eb-69b83a2c1bff", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:10 GMT"} })
14872025-04-30T06:14:10.850ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
14882025-04-30T06:14:10.851ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "02ea084d-c38e-48ea-a6fb-ec49a5e232b6", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:10 GMT"} })
14892025-04-30T06:14:10.851ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
14902025-04-30T06:14:10.852ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "434e9300-c3cc-416b-b701-da671ece8ca8", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:10 GMT"} })
14912025-04-30T06:14:10.852ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
14922025-04-30T06:14:10.853ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1c4e9a3e-eecb-4528-9199-7ca8fe58e821", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:10 GMT"} })
14932025-04-30T06:14:10.853ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1
14942025-04-30T06:14:10.854ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9e521417-d54d-49f6-a589-9189adecd276", "content-length": "734", "date": "Wed, 30 Apr 2025 06:14:10 GMT"} })
14952025-04-30T06:14:10.854ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=0
14962025-04-30T06:14:10.869ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "58c2ac43-0643-4cdf-b342-0fbc7fbf4578", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:10 GMT"} })
14972025-04-30T06:14:10.869ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=1
14982025-04-30T06:14:10.872ZINFOtest_basic (InternalApi): Inserted networking records for external_dns service
14992025-04-30T06:14:10.872ZINFOtest_basic (InternalApi): Inserted service networking records
15002025-04-30T06:14:10.872ZINFOtest_basic (InternalApi): Inserted physical disks
15012025-04-30T06:14:10.872ZINFOtest_basic (InternalApi): Inserted zpools
15022025-04-30T06:14:10.872ZINFOtest_basic (InternalApi): Inserted datasets
15032025-04-30T06:14:10.888ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "77e2c3df-3329-458e-93f0-fc8cf94d9c64", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:10 GMT"} })
15042025-04-30T06:14:10.889ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=0
15052025-04-30T06:14:10.890ZINFOtest_basic (InternalApi): Populated DNS tables for internal DNS
15062025-04-30T06:14:10.893ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4c47f67b-0ba3-4d16-ba41-55c07a94a29a", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:10 GMT"} })
15072025-04-30T06:14:10.893ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=1
15082025-04-30T06:14:10.895ZINFOtest_basic (InternalApi): Populated DNS tables for external DNS
15092025-04-30T06:14:10.896ZTRACtest_basic (InternalApi): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
15102025-04-30T06:14:10.896ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
15112025-04-30T06:14:10.896ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15122025-04-30T06:14:10.896ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
15132025-04-30T06:14:10.897ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15142025-04-30T06:14:10.900ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15152025-04-30T06:14:10.907ZDEBGtest_basic (InternalApi): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
15162025-04-30T06:14:10.908ZTRACtest_basic (InternalApi): authorize begin action = ModifyPolicy actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
15172025-04-30T06:14:10.908ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
15182025-04-30T06:14:10.908ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15192025-04-30T06:14:10.909ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
15202025-04-30T06:14:10.911ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15212025-04-30T06:14:10.915ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15222025-04-30T06:14:10.921ZDEBGtest_basic (InternalApi): authorize result action = ModifyPolicy actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
15232025-04-30T06:14:10.921ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7cdf551b-564a-432b-bb8b-07833946f9ad", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:10 GMT"} })
15242025-04-30T06:14:10.922ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=0
15252025-04-30T06:14:10.926ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8178aa45-c21e-4aed-ba4c-d9681790c321", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:10 GMT"} })
15262025-04-30T06:14:10.926ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=1
15272025-04-30T06:14:10.931ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d927ec6e-282c-4083-bde8-e8976069c8ac", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:10 GMT"} })
15282025-04-30T06:14:10.931ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cmpa
15292025-04-30T06:14:10.932ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b1b4643d-6f8a-4c08-9c79-4ae8ae55980d", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:10 GMT"} })
15302025-04-30T06:14:10.933ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
15312025-04-30T06:14:10.934ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0fd171a8-fb35-44d7-b5cc-6517d8ed95c9", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:10 GMT"} })
15322025-04-30T06:14:10.934ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
15332025-04-30T06:14:10.936ZTRACtest_basic (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = DnsConfig
15342025-04-30T06:14:10.936ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
15352025-04-30T06:14:10.936ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15362025-04-30T06:14:10.936ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
15372025-04-30T06:14:10.937ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15382025-04-30T06:14:10.939ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "feb4db86-6d84-4715-a72a-6e12fd7e816c", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:10 GMT"} })
15392025-04-30T06:14:10.940ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
15402025-04-30T06:14:10.942ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "29202bcd-4a01-426b-b190-01ed205ef739", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:10 GMT"} })
15412025-04-30T06:14:10.942ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0
15422025-04-30T06:14:10.943ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15432025-04-30T06:14:10.958ZDEBGtest_basic (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = DnsConfig result = Ok(())
15442025-04-30T06:14:10.958ZTRACtest_basic (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = DnsConfig
15452025-04-30T06:14:10.959ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
15462025-04-30T06:14:10.959ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15472025-04-30T06:14:10.959ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
15482025-04-30T06:14:10.962ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15492025-04-30T06:14:10.966ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c1674e2a-c53b-4759-89c4-44593e7ad6d1", "content-length": "723", "date": "Wed, 30 Apr 2025 06:14:10 GMT"} })
15502025-04-30T06:14:10.978ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=0
15512025-04-30T06:14:10.982ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15522025-04-30T06:14:10.990ZDEBGtest_basic (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = DnsConfig result = Ok(())
15532025-04-30T06:14:10.995ZTRACtest_basic (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = DnsConfig
15542025-04-30T06:14:10.995ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
15552025-04-30T06:14:10.995ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15562025-04-30T06:14:10.995ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
15572025-04-30T06:14:10.996ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15582025-04-30T06:14:11.003ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "942fbe29-62c4-4da4-92bc-c423c6f30156", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:10 GMT"} })
15592025-04-30T06:14:11.003ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=1
15602025-04-30T06:14:11.004ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15612025-04-30T06:14:11.012ZDEBGtest_basic (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = DnsConfig result = Ok(())
15622025-04-30T06:14:11.035ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6165dcaf-7eb8-4b4c-b56e-4927e9eefb4e", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:10 GMT"} })
15632025-04-30T06:14:11.035ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=0
15642025-04-30T06:14:11.067ZINFOtest_basic (InternalApi): Created recovery silo
15652025-04-30T06:14:11.068ZINFOtest_basic (InternalApi): Created recovery user
15662025-04-30T06:14:11.069ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2ecb12bc-783a-4497-b494-ee88e785d181", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:10 GMT"} })
15672025-04-30T06:14:11.069ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=1
15682025-04-30T06:14:11.073ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5dad61e5-5d01-4875-9507-086335cd636d", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:10 GMT"} })
15692025-04-30T06:14:11.073ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=0
15702025-04-30T06:14:11.076ZINFOtest_basic (InternalApi): Created recovery user's password
15712025-04-30T06:14:11.076ZTRACtest_basic (InternalApi): authorize begin action = ModifyPolicy actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Silo { parent: Fleet, key: 96568163-7df0-4ae3-adb5-89dafc8f25cf, lookup_type: ById(96568163-7df0-4ae3-adb5-89dafc8f25cf) }
15722025-04-30T06:14:11.077ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 96568163-7df0-4ae3-adb5-89dafc8f25cf resource_type = Silo
15732025-04-30T06:14:11.077ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15742025-04-30T06:14:11.077ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
15752025-04-30T06:14:11.081ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15762025-04-30T06:14:11.084ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
15772025-04-30T06:14:11.084ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15782025-04-30T06:14:11.084ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
15792025-04-30T06:14:11.085ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15802025-04-30T06:14:11.091ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15812025-04-30T06:14:11.098ZDEBGtest_basic (InternalApi): authorize result action = ModifyPolicy actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Silo { parent: Fleet, key: 96568163-7df0-4ae3-adb5-89dafc8f25cf, lookup_type: ById(96568163-7df0-4ae3-adb5-89dafc8f25cf) } result = Ok(())
15822025-04-30T06:14:11.098ZDEBGtest_basic (InternalApi): Generated role assignment queries
15832025-04-30T06:14:11.110ZINFOtest_basic (InternalApi): Granted Silo privileges
15842025-04-30T06:14:11.111ZTRACtest_basic (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
15852025-04-30T06:14:11.112ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
15862025-04-30T06:14:11.112ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15872025-04-30T06:14:11.112ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
15882025-04-30T06:14:11.113ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15892025-04-30T06:14:11.122ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15902025-04-30T06:14:11.128ZDEBGtest_basic (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
15912025-04-30T06:14:11.133ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7b678326-dfc6-4e74-885c-e8ae5a1693b9", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:10 GMT"} })
15922025-04-30T06:14:11.134ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=1
15932025-04-30T06:14:11.144ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15942025-04-30T06:14:11.144ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
15952025-04-30T06:14:11.146ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15962025-04-30T06:14:11.154ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15972025-04-30T06:14:11.154ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
15982025-04-30T06:14:11.155ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15992025-04-30T06:14:11.185ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16002025-04-30T06:14:11.185ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
16012025-04-30T06:14:11.189ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16022025-04-30T06:14:11.194ZTRACtest_basic (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
16032025-04-30T06:14:11.195ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
16042025-04-30T06:14:11.205ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16052025-04-30T06:14:11.206ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
16062025-04-30T06:14:11.207ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16072025-04-30T06:14:11.211ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7ffcdf83-f610-46f3-9ff4-1205a7cef13d", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:11 GMT"} })
16082025-04-30T06:14:11.211ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cmpa
16092025-04-30T06:14:11.212ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
16102025-04-30T06:14:11.212ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16112025-04-30T06:14:11.212ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
16122025-04-30T06:14:11.215ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16132025-04-30T06:14:11.218ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6cf9a62a-27e3-4a62-b048-f9e7fa5e0051", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:11 GMT"} })
16142025-04-30T06:14:11.218ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
16152025-04-30T06:14:11.224ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16162025-04-30T06:14:11.229ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16172025-04-30T06:14:11.230ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
16182025-04-30T06:14:11.238ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16192025-04-30T06:14:11.238ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7de66821-3bca-4525-85e2-e3c17a12bba0", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:11 GMT"} })
16202025-04-30T06:14:11.239ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
16212025-04-30T06:14:11.245ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8634e8a1-0c94-49de-8117-cc6a41ac7849", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:11 GMT"} })
16222025-04-30T06:14:11.246ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
16232025-04-30T06:14:11.246ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16242025-04-30T06:14:11.262ZDEBGtest_basic (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
16252025-04-30T06:14:11.262ZTRACtest_basic (InternalApi): successful lookup of siloed resource "Vpc" using built-in user
16262025-04-30T06:14:11.263ZTRACtest_basic (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
16272025-04-30T06:14:11.263ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
16282025-04-30T06:14:11.263ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16292025-04-30T06:14:11.263ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
16302025-04-30T06:14:11.268ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16312025-04-30T06:14:11.278ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
16322025-04-30T06:14:11.278ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16332025-04-30T06:14:11.278ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
16342025-04-30T06:14:11.279ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16352025-04-30T06:14:11.279ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e295e069-5630-4cf8-b470-2b7169ca4583", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:11 GMT"} })
16362025-04-30T06:14:11.279ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1
16372025-04-30T06:14:11.299ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16382025-04-30T06:14:11.299ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16392025-04-30T06:14:11.299ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
16402025-04-30T06:14:11.301ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16412025-04-30T06:14:11.303ZTRACtest_basic (dns): buffer buffer = Length: 79 (0x4f) bytes\n0000: 60 0e 01 00 00 01 00 00 00 00 00 01 10 5f 63 72 `............_cr\n0010: 75 63 69 62 6c 65 2d 70 61 6e 74 72 79 04 5f 74 ucible-pantry._t\n0020: 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 cp.control-plane\n0030: 05 6f 78 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 .oxide.internal.\n0040: 00 21 00 01 00 00 29 04 d0 00 00 00 00 00 00 .!....)........ kind = dns peer_addr = [::1]:54632 req_id = 9439641e-79a5-47bc-98a0-2af9e827636a
16422025-04-30T06:14:11.303ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 24590,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_crucible-pantry._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_crucible-pantry._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 16,\n 95,\n 99,\n 114,\n 117,\n 99,\n 105,\n 98,\n 108,\n 101,\n 45,\n 112,\n 97,\n 110,\n 116,\n 114,\n 121,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54632 req_id = 9439641e-79a5-47bc-98a0-2af9e827636a
16432025-04-30T06:14:11.304ZDEBGtest_basic (store): query key key = _crucible-pantry._tcp
16442025-04-30T06:14:11.305ZDEBGtest_basic (store): query key key = 6880dc28-d971-4307-bf8a-dd2d981a337c.host
16452025-04-30T06:14:11.305ZDEBGtest_basic (dns): dns response additional_records = [Record { name_labels: Name("6880dc28-d971-4307-bf8a-dd2d981a337c.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:54632 query = LowerQuery { name: LowerName(Name("_crucible-pantry._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_crucible-pantry._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_crucible-pantry._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 43262, target: Name("6880dc28-d971-4307-bf8a-dd2d981a337c.host.control-plane.oxide.internal") })) }] req_id = 9439641e-79a5-47bc-98a0-2af9e827636a
16462025-04-30T06:14:11.307ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16472025-04-30T06:14:11.315ZDEBGtest_basic (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
16482025-04-30T06:14:11.315ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16492025-04-30T06:14:11.315ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
16502025-04-30T06:14:11.315ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16512025-04-30T06:14:11.317ZTRACtest_basic (dns): buffer buffer = Length: 99 (0x63) bytes\n0000: 08 47 01 00 00 01 00 00 00 00 00 01 24 36 38 38 .G..........$688\n0010: 30 64 63 32 38 2d 64 39 37 31 2d 34 33 30 37 2d 0dc28-d971-4307-\n0020: 62 66 38 61 2d 64 64 32 64 39 38 31 61 33 33 37 bf8a-dd2d981a337\n0030: 63 04 68 6f 73 74 0d 63 6f 6e 74 72 6f 6c 2d 70 c.host.control-p\n0040: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0050: 6e 61 6c 00 00 1c 00 01 00 00 29 04 d0 00 00 00 nal.......).....\n0060: 00 00 00 ... kind = dns peer_addr = [::1]:61496 req_id = ee3c3c09-176a-4aac-b129-1e3dca5c3f87
16522025-04-30T06:14:11.317ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 2119,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("6880dc28-d971-4307-bf8a-dd2d981a337c.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("6880dc28-d971-4307-bf8a-dd2d981a337c.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 36,\n 54,\n 56,\n 56,\n 48,\n 100,\n 99,\n 50,\n 56,\n 45,\n 100,\n 57,\n 55,\n 49,\n 45,\n 52,\n 51,\n 48,\n 55,\n 45,\n 98,\n 102,\n 56,\n 97,\n 45,\n 100,\n 100,\n 50,\n 100,\n 57,\n 56,\n 49,\n 97,\n 51,\n 51,\n 55,\n 99,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:61496 req_id = ee3c3c09-176a-4aac-b129-1e3dca5c3f87
16532025-04-30T06:14:11.318ZDEBGtest_basic (store): query key key = 6880dc28-d971-4307-bf8a-dd2d981a337c.host
16542025-04-30T06:14:11.318ZDEBGtest_basic (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:61496 query = LowerQuery { name: LowerName(Name("6880dc28-d971-4307-bf8a-dd2d981a337c.host.control-plane.oxide.internal.")), original: Query { name: Name("6880dc28-d971-4307-bf8a-dd2d981a337c.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("6880dc28-d971-4307-bf8a-dd2d981a337c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = ee3c3c09-176a-4aac-b129-1e3dca5c3f87
16552025-04-30T06:14:11.318ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa008a43-a4f2-4dd4-a802-e9a7fb6eaa9d", "content-length": "723", "date": "Wed, 30 Apr 2025 06:14:11 GMT"} })
16562025-04-30T06:14:11.319ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=0
16572025-04-30T06:14:11.550ZINFOtest_basic (dropshot): accepted connection kind = pantry local_addr = [::1]:43262 remote_addr = [::1]:65235 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
16582025-04-30T06:14:11.558ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16592025-04-30T06:14:11.558ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
16602025-04-30T06:14:11.559ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16612025-04-30T06:14:11.559ZTRACtest_basic (dropshot): incoming request kind = pantry local_addr = [::1]:43262 method = GET remote_addr = [::1]:65235 req_id = 59c459ca-ecd0-434a-92ed-f4d7f5f11360 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
16622025-04-30T06:14:11.560ZINFOtest_basic (dropshot): request completed kind = pantry latency_us = 1110 local_addr = [::1]:43262 method = GET remote_addr = [::1]:65235 req_id = 59c459ca-ecd0-434a-92ed-f4d7f5f11360 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
16632025-04-30T06:14:12.290ZINFOtest_basic (dropshot): accepted connection kind = pantry local_addr = [::1]:43262 remote_addr = [::1]:58454 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
16642025-04-30T06:14:12.290ZINFOtest_basic (dropshot): accepted connection kind = pantry local_addr = [::1]:43262 remote_addr = [::1]:55637 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
16652025-04-30T06:14:12.290ZINFOtest_basic (dropshot): accepted connection kind = pantry local_addr = [::1]:43262 remote_addr = [::1]:46090 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
16662025-04-30T06:14:12.290ZINFOtest_basic (dropshot): accepted connection kind = pantry local_addr = [::1]:43262 remote_addr = [::1]:60754 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
16672025-04-30T06:14:12.290ZINFOtest_basic (dropshot): accepted connection kind = pantry local_addr = [::1]:43262 remote_addr = [::1]:36419 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
16682025-04-30T06:14:12.291ZINFOtest_basic (dropshot): accepted connection kind = pantry local_addr = [::1]:43262 remote_addr = [::1]:45211 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
16692025-04-30T06:14:12.296ZINFOtest_basic (dropshot): accepted connection kind = pantry local_addr = [::1]:43262 remote_addr = [::1]:53619 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
16702025-04-30T06:14:12.305ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16712025-04-30T06:14:12.306ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
16722025-04-30T06:14:12.308ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16732025-04-30T06:14:12.310ZTRACtest_basic (dropshot): incoming request kind = pantry local_addr = [::1]:43262 method = GET remote_addr = [::1]:58454 req_id = 4fc66bfb-74d2-4ad0-88b5-0e54445901f4 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
16742025-04-30T06:14:12.313ZTRACtest_basic (dropshot): incoming request kind = pantry local_addr = [::1]:43262 method = GET remote_addr = [::1]:55637 req_id = 0cfc8036-c3a7-4b78-84e5-5b114d042a60 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
16752025-04-30T06:14:12.313ZTRACtest_basic (dropshot): incoming request kind = pantry local_addr = [::1]:43262 method = GET remote_addr = [::1]:46090 req_id = 7ed72f4d-552f-41d5-acac-32553b79a42f sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
16762025-04-30T06:14:12.316ZTRACtest_basic (dropshot): incoming request kind = pantry local_addr = [::1]:43262 method = GET remote_addr = [::1]:60754 req_id = e1fd83ca-5364-40b6-ba5c-ca1e18c66af3 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
16772025-04-30T06:14:12.316ZTRACtest_basic (dropshot): incoming request kind = pantry local_addr = [::1]:43262 method = GET remote_addr = [::1]:36419 req_id = eb31e511-10ec-4c25-90b0-93c910609bbe sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
16782025-04-30T06:14:12.316ZTRACtest_basic (dropshot): incoming request kind = pantry local_addr = [::1]:43262 method = GET remote_addr = [::1]:45211 req_id = 0a21d655-89bc-49c4-b9ef-c8be48b69a80 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
16792025-04-30T06:14:12.317ZTRACtest_basic (dropshot): incoming request kind = pantry local_addr = [::1]:43262 method = GET remote_addr = [::1]:53619 req_id = e9362d1c-42fc-4e65-831a-e5276060e7bd sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
16802025-04-30T06:14:12.318ZINFOtest_basic (dropshot): request completed kind = pantry latency_us = 7771 local_addr = [::1]:43262 method = GET remote_addr = [::1]:58454 req_id = 4fc66bfb-74d2-4ad0-88b5-0e54445901f4 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
16812025-04-30T06:14:12.318ZINFOtest_basic (dropshot): request completed kind = pantry latency_us = 5431 local_addr = [::1]:43262 method = GET remote_addr = [::1]:55637 req_id = 0cfc8036-c3a7-4b78-84e5-5b114d042a60 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
16822025-04-30T06:14:12.319ZINFOtest_basic (dropshot): request completed kind = pantry latency_us = 5628 local_addr = [::1]:43262 method = GET remote_addr = [::1]:46090 req_id = 7ed72f4d-552f-41d5-acac-32553b79a42f response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
16832025-04-30T06:14:12.319ZINFOtest_basic (dropshot): request completed kind = pantry latency_us = 3059 local_addr = [::1]:43262 method = GET remote_addr = [::1]:60754 req_id = e1fd83ca-5364-40b6-ba5c-ca1e18c66af3 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
16842025-04-30T06:14:12.319ZINFOtest_basic (dropshot): request completed kind = pantry latency_us = 2607 local_addr = [::1]:43262 method = GET remote_addr = [::1]:36419 req_id = eb31e511-10ec-4c25-90b0-93c910609bbe response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
16852025-04-30T06:14:12.319ZINFOtest_basic (dropshot): request completed kind = pantry latency_us = 2529 local_addr = [::1]:43262 method = GET remote_addr = [::1]:45211 req_id = 0a21d655-89bc-49c4-b9ef-c8be48b69a80 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
16862025-04-30T06:14:12.319ZINFOtest_basic (dropshot): request completed kind = pantry latency_us = 2489 local_addr = [::1]:43262 method = GET remote_addr = [::1]:53619 req_id = e9362d1c-42fc-4e65-831a-e5276060e7bd response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
16872025-04-30T06:14:12.337ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16882025-04-30T06:14:12.337ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
16892025-04-30T06:14:12.338ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16902025-04-30T06:14:12.355ZTRACtest_basic (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
16912025-04-30T06:14:12.356ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
16922025-04-30T06:14:12.356ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16932025-04-30T06:14:12.356ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
16942025-04-30T06:14:12.359ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16952025-04-30T06:14:12.383ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
16962025-04-30T06:14:12.383ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16972025-04-30T06:14:12.384ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
16982025-04-30T06:14:12.387ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16992025-04-30T06:14:12.396ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
17002025-04-30T06:14:12.397ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17012025-04-30T06:14:12.397ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
17022025-04-30T06:14:12.402ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17032025-04-30T06:14:12.413ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17042025-04-30T06:14:12.433ZDEBGtest_basic (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
17052025-04-30T06:14:12.433ZTRACtest_basic (InternalApi): successful lookup of siloed resource "Vpc" using built-in user
17062025-04-30T06:14:12.433ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17072025-04-30T06:14:12.433ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
17082025-04-30T06:14:12.435ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17092025-04-30T06:14:12.448ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b70f98e7-6a9b-41c7-ab31-1451c8d82e12", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:12 GMT"} })
17102025-04-30T06:14:12.449ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=1
17112025-04-30T06:14:12.449ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17122025-04-30T06:14:12.449ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
17132025-04-30T06:14:12.450ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17142025-04-30T06:14:12.470ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17152025-04-30T06:14:12.471ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
17162025-04-30T06:14:12.473ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17172025-04-30T06:14:12.483ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17182025-04-30T06:14:12.483ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
17192025-04-30T06:14:12.484ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17202025-04-30T06:14:12.489ZTRACtest_basic (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") }
17212025-04-30T06:14:12.489ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
17222025-04-30T06:14:12.490ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17232025-04-30T06:14:12.490ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
17242025-04-30T06:14:12.491ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17252025-04-30T06:14:12.494ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
17262025-04-30T06:14:12.495ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17272025-04-30T06:14:12.495ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
17282025-04-30T06:14:12.496ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17292025-04-30T06:14:12.497ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fab41e68-1f75-4f15-969e-702764611b82", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:12 GMT"} })
17302025-04-30T06:14:12.498ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=0
17312025-04-30T06:14:12.501ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
17322025-04-30T06:14:12.501ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17332025-04-30T06:14:12.501ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
17342025-04-30T06:14:12.502ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17352025-04-30T06:14:12.506ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17362025-04-30T06:14:12.530ZDEBGtest_basic (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
17372025-04-30T06:14:12.530ZTRACtest_basic (InternalApi): successful lookup of siloed resource "VpcSubnet" using built-in user
17382025-04-30T06:14:12.530ZTRACtest_basic (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") }
17392025-04-30T06:14:12.530ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
17402025-04-30T06:14:12.530ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17412025-04-30T06:14:12.530ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
17422025-04-30T06:14:12.536ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17432025-04-30T06:14:12.540ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1fe7f13c-76f2-45d3-9cbf-94c054504a43", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:12 GMT"} })
17442025-04-30T06:14:12.541ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=1
17452025-04-30T06:14:12.542ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
17462025-04-30T06:14:12.542ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17472025-04-30T06:14:12.542ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
17482025-04-30T06:14:12.545ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17492025-04-30T06:14:12.549ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
17502025-04-30T06:14:12.550ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17512025-04-30T06:14:12.550ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
17522025-04-30T06:14:12.551ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17532025-04-30T06:14:12.554ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17542025-04-30T06:14:12.570ZDEBGtest_basic (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
17552025-04-30T06:14:12.570ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17562025-04-30T06:14:12.570ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
17572025-04-30T06:14:12.571ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17582025-04-30T06:14:12.578ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "49956df2-9da0-42e0-a3c7-10155b97ab89", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:12 GMT"} })
17592025-04-30T06:14:12.578ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=0
17602025-04-30T06:14:12.582ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17612025-04-30T06:14:12.582ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
17622025-04-30T06:14:12.584ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17632025-04-30T06:14:12.584ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3cf2c70b-67c4-4f5e-9f4c-8e316195bebf", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:12 GMT"} })
17642025-04-30T06:14:12.585ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=1
17652025-04-30T06:14:12.589ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17662025-04-30T06:14:12.589ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
17672025-04-30T06:14:12.590ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17682025-04-30T06:14:12.598ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a060ba00-0bb5-4c30-b54f-2b541f67f0e3", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:12 GMT"} })
17692025-04-30T06:14:12.599ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cmpa
17702025-04-30T06:14:12.599ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17712025-04-30T06:14:12.602ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
17722025-04-30T06:14:12.603ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17732025-04-30T06:14:12.608ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2070c931-675e-4ed3-bc68-0a49e71bdd80", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:12 GMT"} })
17742025-04-30T06:14:12.609ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
17752025-04-30T06:14:12.610ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17762025-04-30T06:14:12.610ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
17772025-04-30T06:14:12.610ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17782025-04-30T06:14:12.612ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "50592692-7124-4202-8e5b-561627378024", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:12 GMT"} })
17792025-04-30T06:14:12.613ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
17802025-04-30T06:14:12.614ZTRACtest_basic (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") }
17812025-04-30T06:14:12.614ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
17822025-04-30T06:14:12.614ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17832025-04-30T06:14:12.614ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
17842025-04-30T06:14:12.615ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17852025-04-30T06:14:12.616ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ac393b4c-2524-44d4-ab6b-bc9990a62135", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:12 GMT"} })
17862025-04-30T06:14:12.616ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
17872025-04-30T06:14:12.618ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
17882025-04-30T06:14:12.618ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17892025-04-30T06:14:12.618ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
17902025-04-30T06:14:12.619ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17912025-04-30T06:14:12.621ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "59373425-3b8d-4039-b11f-39bf47d466fd", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:12 GMT"} })
17922025-04-30T06:14:12.621ZTRACtest_basic (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
17932025-04-30T06:14:12.621ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
17942025-04-30T06:14:12.621ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
17952025-04-30T06:14:12.621ZDEBGtest_basic (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
17962025-04-30T06:14:12.622ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
17972025-04-30T06:14:12.622ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
17982025-04-30T06:14:12.622ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17992025-04-30T06:14:12.622ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
18002025-04-30T06:14:12.623ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18012025-04-30T06:14:12.625ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18022025-04-30T06:14:12.641ZDEBGtest_basic (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
18032025-04-30T06:14:12.641ZTRACtest_basic (InternalApi): successful lookup of siloed resource "VpcSubnet" using built-in user
18042025-04-30T06:14:12.641ZTRACtest_basic (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") }
18052025-04-30T06:14:12.641ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
18062025-04-30T06:14:12.641ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18072025-04-30T06:14:12.641ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
18082025-04-30T06:14:12.642ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18092025-04-30T06:14:12.643ZDEBGtest_basic (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18102025-04-30T06:14:12.651ZDEBGtest_basic (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
18112025-04-30T06:14:12.651ZTRACtest_basic (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
18122025-04-30T06:14:12.651ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
18132025-04-30T06:14:12.651ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
18142025-04-30T06:14:12.651ZDEBGtest_basic (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
18152025-04-30T06:14:12.652ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
18162025-04-30T06:14:12.657ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
18172025-04-30T06:14:12.657ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18182025-04-30T06:14:12.658ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
18192025-04-30T06:14:12.659ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18202025-04-30T06:14:12.660ZDEBGtest_basic (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18212025-04-30T06:14:12.669ZDEBGtest_basic (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
18222025-04-30T06:14:12.669ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
18232025-04-30T06:14:12.669ZDEBGtest_basic (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
18242025-04-30T06:14:12.670ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
18252025-04-30T06:14:12.672ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
18262025-04-30T06:14:12.672ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18272025-04-30T06:14:12.672ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
18282025-04-30T06:14:12.673ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18292025-04-30T06:14:12.819ZDEBGtest_basic (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:42420
18302025-04-30T06:14:12.819ZDEBGtest_basic (BackgroundTasks): client request SledAgent = http://[::1]:42420 background_task = inventory_collection body = None method = GET uri = http://[::1]:42420/inventory
18312025-04-30T06:14:12.821ZINFOtest_basic (dropshot): accepted connection local_addr = [::1]:42420 remote_addr = [::1]:42656 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
18322025-04-30T06:14:12.821ZTRACtest_basic (dropshot): incoming request local_addr = [::1]:42420 method = GET remote_addr = [::1]:42656 req_id = 01968fdc-75e1-4799-bcab-510d2907bfec sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
18332025-04-30T06:14:12.823ZINFOtest_basic (dropshot): request completed latency_us = 1420 local_addr = [::1]:42420 method = GET remote_addr = [::1]:42656 req_id = 01968fdc-75e1-4799-bcab-510d2907bfec response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
18342025-04-30T06:14:12.823ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18352025-04-30T06:14:12.839ZDEBGtest_basic (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
18362025-04-30T06:14:12.840ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18372025-04-30T06:14:12.840ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
18382025-04-30T06:14:12.840ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18392025-04-30T06:14:12.841ZDEBGtest_basic (BackgroundTasks): client response SledAgent = http://[::1]:42420 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:42420/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "01968fdc-75e1-4799-bcab-510d2907bfec", "content-length": "2769", "date": "Wed, 30 Apr 2025 06:14:12 GMT"} })
18402025-04-30T06:14:12.846ZDEBGtest_basic (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
18412025-04-30T06:14:12.846ZDEBGtest_basic (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
18422025-04-30T06:14:12.846ZDEBGtest_basic (BackgroundTasks): finished collection background_task = inventory_collection
18432025-04-30T06:14:12.849ZTRACtest_basic (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
18442025-04-30T06:14:12.849ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
18452025-04-30T06:14:12.849ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
18462025-04-30T06:14:12.849ZDEBGtest_basic (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
18472025-04-30T06:14:12.853ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
18482025-04-30T06:14:12.864ZDEBGtest_basic (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18492025-04-30T06:14:12.874ZDEBGtest_basic (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
18502025-04-30T06:14:12.875ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
18512025-04-30T06:14:12.875ZDEBGtest_basic (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
18522025-04-30T06:14:12.876ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
18532025-04-30T06:14:12.883ZDEBGtest_basic (nexus): resolved names for firewall rules instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 53d2e1ed-df5d-4f37-aa3e-e472d5878035, kind: Service { id: 088ac96c-981b-4054-bea6-d8ab23ae1366 }, name: Name("nexus-088ac96c-981b-4054-bea6-d8ab23ae1366"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 1])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("external-dns"): [NetworkInterface { id: a5b14f67-66ad-4f05-bc8e-859cf15ab7f4, kind: Service { id: c4e9f997-f271-4a58-92bc-01625e84ce01 }, name: Name("external-dns-c4e9f997-f271-4a58-92bc-01625e84ce01"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 0])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
18542025-04-30T06:14:12.883ZTRACtest_basic (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
18552025-04-30T06:14:12.884ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
18562025-04-30T06:14:12.884ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18572025-04-30T06:14:12.884ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
18582025-04-30T06:14:12.886ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18592025-04-30T06:14:12.889ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18602025-04-30T06:14:12.902ZDEBGtest_basic (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
18612025-04-30T06:14:12.902ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18622025-04-30T06:14:12.903ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
18632025-04-30T06:14:12.905ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18642025-04-30T06:14:12.912ZTRACtest_basic (nexus): fetched allowlist from DB allowed = AllowList { id: 001de000-a110-4000-8000-000000000000, time_created: 2025-04-30T06:06:08.248500Z, time_modified: 2025-04-30T06:14:11.128781Z, allowed_ips: None }
18652025-04-30T06:14:12.912ZDEBGtest_basic (nexus): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified.
18662025-04-30T06:14:12.913ZDEBGtest_basic (nexus): resolved firewall rules for sled agents sled_agent_rules = [ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: a5b14f67-66ad-4f05-bc8e-859cf15ab7f4, kind: Service { id: c4e9f997-f271-4a58-92bc-01625e84ce01 }, name: Name("external-dns-c4e9f997-f271-4a58-92bc-01625e84ce01"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 0])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }, ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 53d2e1ed-df5d-4f37-aa3e-e472d5878035, kind: Service { id: 088ac96c-981b-4054-bea6-d8ab23ae1366 }, name: Name("nexus-088ac96c-981b-4054-bea6-d8ab23ae1366"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 1])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }]
18672025-04-30T06:14:12.913ZDEBGtest_basic (nexus): resolved 2 rules for sleds
18682025-04-30T06:14:13.091ZDEBGtest_basic (nexus): resolved sleds for vpc oxide-services vpc_to_sled = [Sled { identity: SledIdentity { id: b6d65341-167c-41df-9b5c-41cded99c229, time_created: 2025-04-30T06:14:03.570056Z, time_modified: 2025-04-30T06:14:03.570056Z }, time_deleted: None, rcgen: Generation(Generation(1)), rack_id: c19a698f-c6f9-4a17-ae30-20d711b8f7dc, is_scrimlet: true, serial_number: "sim-b6d65341", part_number: "Unknown", revision: SqlU32(0), usable_hardware_threads: SqlU32(16), usable_physical_ram: ByteCount(ByteCount(34359738368)), reservoir_size: ByteCount(ByteCount(17179869184)), ip: ::1, port: SqlU16(42420), last_used_address: ::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(51632) }]
18692025-04-30T06:14:13.091ZDEBGtest_basic (nexus): sending firewall rules to sled agents
18702025-04-30T06:14:13.091ZTRACtest_basic (InstanceAllocator): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database
18712025-04-30T06:14:13.091ZDEBGtest_basic (InstanceAllocator): roles roles = RoleSet { roles: {} }
18722025-04-30T06:14:13.093ZDEBGtest_basic (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
18732025-04-30T06:14:13.102ZTRACtest_basic (InstanceAllocator): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Sled { parent: Fleet, key: b6d65341-167c-41df-9b5c-41cded99c229, lookup_type: ById(b6d65341-167c-41df-9b5c-41cded99c229) }
18742025-04-30T06:14:13.103ZTRACtest_basic (InstanceAllocator): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
18752025-04-30T06:14:13.103ZTRACtest_basic (InstanceAllocator): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database
18762025-04-30T06:14:13.103ZDEBGtest_basic (InstanceAllocator): roles roles = RoleSet { roles: {} }
18772025-04-30T06:14:13.104ZDEBGtest_basic (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
18782025-04-30T06:14:13.114ZDEBGtest_basic (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
18792025-04-30T06:14:13.120ZDEBGtest_basic (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Sled { parent: Fleet, key: b6d65341-167c-41df-9b5c-41cded99c229, lookup_type: ById(b6d65341-167c-41df-9b5c-41cded99c229) } result = Ok(())
18802025-04-30T06:14:13.197ZDEBGtest_basic (nexus): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 body = Some(Body) method = PUT uri = http://[::1]:42420/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
18812025-04-30T06:14:13.199ZINFOtest_basic (dropshot): accepted connection local_addr = [::1]:42420 remote_addr = [::1]:51044 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
18822025-04-30T06:14:13.208ZTRACtest_basic (dropshot): incoming request local_addr = [::1]:42420 method = PUT remote_addr = [::1]:51044 req_id = 69460e0b-c17d-4cac-a823-f1e451fe60dd sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
18832025-04-30T06:14:13.215ZINFOtest_basic (dropshot): request completed latency_us = 6623 local_addr = [::1]:42420 method = PUT remote_addr = [::1]:51044 req_id = 69460e0b-c17d-4cac-a823-f1e451fe60dd response_code = 204 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
18842025-04-30T06:14:13.222ZDEBGtest_basic (nexus): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 result = Ok(Response { url: "http://[::1]:42420/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "69460e0b-c17d-4cac-a823-f1e451fe60dd", "date": "Wed, 30 Apr 2025 06:14:12 GMT"} })
18852025-04-30T06:14:13.226ZINFOtest_basic (nexus): updated firewall rules on 1 sleds
18862025-04-30T06:14:13.226ZTRACtest_basic (ServiceBalancer): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database
18872025-04-30T06:14:13.226ZDEBGtest_basic (ServiceBalancer): roles roles = RoleSet { roles: {} }
18882025-04-30T06:14:13.227ZDEBGtest_basic (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
18892025-04-30T06:14:13.229ZDEBGtest_basic (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 2 reason = Signaled
18902025-04-30T06:14:13.230ZDEBGtest_basic (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
18912025-04-30T06:14:13.230ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
18922025-04-30T06:14:13.230ZDEBGtest_basic (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
18932025-04-30T06:14:13.233ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
18942025-04-30T06:14:13.234ZDEBGtest_basic (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 2 reason = Signaled
18952025-04-30T06:14:13.234ZTRACtest_basic (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal
18962025-04-30T06:14:13.234ZDEBGtest_basic (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 2 reason = Signaled
18972025-04-30T06:14:13.234ZDEBGtest_basic (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
18982025-04-30T06:14:13.234ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
18992025-04-30T06:14:13.234ZDEBGtest_basic (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
19002025-04-30T06:14:13.235ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
19012025-04-30T06:14:13.235ZDEBGtest_basic (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 2 reason = Signaled
19022025-04-30T06:14:13.235ZTRACtest_basic (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal
19032025-04-30T06:14:13.235ZDEBGtest_basic (BackgroundTasks): activating background_task = external_endpoints iteration = 2 reason = Signaled
19042025-04-30T06:14:13.235ZTRACtest_basic (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet
19052025-04-30T06:14:13.235ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = external_endpoints resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19062025-04-30T06:14:13.235ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
19072025-04-30T06:14:13.235ZDEBGtest_basic (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
19082025-04-30T06:14:13.236ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
19092025-04-30T06:14:13.258ZTRACtest_basic (dns): buffer buffer = Length: 75 (0x4b) bytes\n0000: 14 82 01 00 00 01 00 00 00 00 00 01 0c 5f 6e 61 ............._na\n0010: 6d 65 73 65 72 76 69 63 65 04 5f 74 63 70 0d 63 meservice._tcp.c\n0020: 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 ontrol-plane.oxi\n0030: 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 de.internal..!..\n0040: 00 00 29 04 d0 00 00 00 00 00 00 ..)........ kind = dns peer_addr = [::1]:62624 req_id = a9e1477d-f1de-4381-82f9-7e469e5abf23
19102025-04-30T06:14:13.260ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 5250,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_nameservice._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_nameservice._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 12,\n 95,\n 110,\n 97,\n 109,\n 101,\n 115,\n 101,\n 114,\n 118,\n 105,\n 99,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62624 req_id = a9e1477d-f1de-4381-82f9-7e469e5abf23
19112025-04-30T06:14:13.260ZDEBGtest_basic (store): query key key = _nameservice._tcp
19122025-04-30T06:14:13.260ZDEBGtest_basic (store): query key key = bab7bfc0-bde9-4584-a405-df241034d9fc.host
19132025-04-30T06:14:13.260ZDEBGtest_basic (dns): dns response additional_records = [Record { name_labels: Name("bab7bfc0-bde9-4584-a405-df241034d9fc.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:62624 query = LowerQuery { name: LowerName(Name("_nameservice._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 57818, target: Name("bab7bfc0-bde9-4584-a405-df241034d9fc.host.control-plane.oxide.internal") })) }] req_id = a9e1477d-f1de-4381-82f9-7e469e5abf23
19142025-04-30T06:14:13.261ZTRACtest_basic (dns): buffer buffer = Length: 76 (0x4c) bytes\n0000: 8e 10 01 00 00 01 00 00 00 00 00 01 0d 5f 65 78 ............._ex\n0010: 74 65 72 6e 61 6c 2d 64 6e 73 04 5f 74 63 70 0d ternal-dns._tcp.\n0020: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0030: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 21 00 ide.internal..!.\n0040: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:50489 req_id = 124de61b-15f1-4bf0-9aac-5f34b941d10c
19152025-04-30T06:14:13.261ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 36368,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_external-dns._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_external-dns._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 13,\n 95,\n 101,\n 120,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 45,\n 100,\n 110,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:50489 req_id = 124de61b-15f1-4bf0-9aac-5f34b941d10c
19162025-04-30T06:14:13.261ZDEBGtest_basic (store): query key key = _external-dns._tcp
19172025-04-30T06:14:13.262ZDEBGtest_basic (store): query key key = c4e9f997-f271-4a58-92bc-01625e84ce01.host
19182025-04-30T06:14:13.262ZDEBGtest_basic (dns): dns response additional_records = [Record { name_labels: Name("c4e9f997-f271-4a58-92bc-01625e84ce01.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:50489 query = LowerQuery { name: LowerName(Name("_external-dns._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 46146, target: Name("c4e9f997-f271-4a58-92bc-01625e84ce01.host.control-plane.oxide.internal") })) }] req_id = 124de61b-15f1-4bf0-9aac-5f34b941d10c
19192025-04-30T06:14:13.262ZTRACtest_basic (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig
19202025-04-30T06:14:13.263ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_internal dns_group = internal resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19212025-04-30T06:14:13.264ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
19222025-04-30T06:14:13.264ZDEBGtest_basic (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
19232025-04-30T06:14:13.266ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
19242025-04-30T06:14:13.266ZTRACtest_basic (ServiceBalancer): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: c19a698f-c6f9-4a17-ae30-20d711b8f7dc, lookup_type: ById(c19a698f-c6f9-4a17-ae30-20d711b8f7dc) }
19252025-04-30T06:14:13.266ZTRACtest_basic (ServiceBalancer): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19262025-04-30T06:14:13.266ZTRACtest_basic (ServiceBalancer): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database
19272025-04-30T06:14:13.266ZDEBGtest_basic (ServiceBalancer): roles roles = RoleSet { roles: {} }
19282025-04-30T06:14:13.269ZDEBGtest_basic (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
19292025-04-30T06:14:13.269ZDEBGtest_basic (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 57818, target: Name("bab7bfc0-bde9-4584-a405-df241034d9fc.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("bab7bfc0-bde9-4584-a405-df241034d9fc.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2518, tv_nsec: 68399799 } })
19302025-04-30T06:14:13.269ZTRACtest_basic (DnsResolver): lookup_service_targets: looking up SRV target
19312025-04-30T06:14:13.269ZDEBGtest_basic (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 46146, target: Name("c4e9f997-f271-4a58-92bc-01625e84ce01.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("c4e9f997-f271-4a58-92bc-01625e84ce01.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2518, tv_nsec: 68799495 } })
19322025-04-30T06:14:13.269ZTRACtest_basic (DnsResolver): lookup_service_targets: looking up SRV target
19332025-04-30T06:14:13.271ZTRACtest_basic (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig
19342025-04-30T06:14:13.271ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_external dns_group = external resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19352025-04-30T06:14:13.271ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
19362025-04-30T06:14:13.271ZDEBGtest_basic (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
19372025-04-30T06:14:13.271ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
19382025-04-30T06:14:13.272ZTRACtest_basic (dns): buffer buffer = Length: 99 (0x63) bytes\n0000: 82 06 01 00 00 01 00 00 00 00 00 01 24 62 61 62 ............$bab\n0010: 37 62 66 63 30 2d 62 64 65 39 2d 34 35 38 34 2d 7bfc0-bde9-4584-\n0020: 61 34 30 35 2d 64 66 32 34 31 30 33 34 64 39 66 a405-df241034d9f\n0030: 63 04 68 6f 73 74 0d 63 6f 6e 74 72 6f 6c 2d 70 c.host.control-p\n0040: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0050: 6e 61 6c 00 00 1c 00 01 00 00 29 04 d0 00 00 00 nal.......).....\n0060: 00 00 00 ... kind = dns peer_addr = [::1]:65277 req_id = d4aa0dc5-6ec5-448e-b0bc-fb67d960c735
19392025-04-30T06:14:13.272ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 33286,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("bab7bfc0-bde9-4584-a405-df241034d9fc.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("bab7bfc0-bde9-4584-a405-df241034d9fc.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 36,\n 98,\n 97,\n 98,\n 55,\n 98,\n 102,\n 99,\n 48,\n 45,\n 98,\n 100,\n 101,\n 57,\n 45,\n 52,\n 53,\n 56,\n 52,\n 45,\n 97,\n 52,\n 48,\n 53,\n 45,\n 100,\n 102,\n 50,\n 52,\n 49,\n 48,\n 51,\n 52,\n 100,\n 57,\n 102,\n 99,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:65277 req_id = d4aa0dc5-6ec5-448e-b0bc-fb67d960c735
19402025-04-30T06:14:13.273ZDEBGtest_basic (store): query key key = bab7bfc0-bde9-4584-a405-df241034d9fc.host
19412025-04-30T06:14:13.273ZDEBGtest_basic (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:65277 query = LowerQuery { name: LowerName(Name("bab7bfc0-bde9-4584-a405-df241034d9fc.host.control-plane.oxide.internal.")), original: Query { name: Name("bab7bfc0-bde9-4584-a405-df241034d9fc.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("bab7bfc0-bde9-4584-a405-df241034d9fc.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = d4aa0dc5-6ec5-448e-b0bc-fb67d960c735
19422025-04-30T06:14:13.275ZTRACtest_basic (dns): buffer buffer = Length: 99 (0x63) bytes\n0000: 7f d0 01 00 00 01 00 00 00 00 00 01 24 63 34 65 ............$c4e\n0010: 39 66 39 39 37 2d 66 32 37 31 2d 34 61 35 38 2d 9f997-f271-4a58-\n0020: 39 32 62 63 2d 30 31 36 32 35 65 38 34 63 65 30 92bc-01625e84ce0\n0030: 31 04 68 6f 73 74 0d 63 6f 6e 74 72 6f 6c 2d 70 1.host.control-p\n0040: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0050: 6e 61 6c 00 00 1c 00 01 00 00 29 04 d0 00 00 00 nal.......).....\n0060: 00 00 00 ... kind = dns peer_addr = [::1]:64542 req_id = d043e99a-450a-406c-8c2a-65cd09fa964a
19432025-04-30T06:14:13.275ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 32720,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("c4e9f997-f271-4a58-92bc-01625e84ce01.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("c4e9f997-f271-4a58-92bc-01625e84ce01.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 36,\n 99,\n 52,\n 101,\n 57,\n 102,\n 57,\n 57,\n 55,\n 45,\n 102,\n 50,\n 55,\n 49,\n 45,\n 52,\n 97,\n 53,\n 56,\n 45,\n 57,\n 50,\n 98,\n 99,\n 45,\n 48,\n 49,\n 54,\n 50,\n 53,\n 101,\n 56,\n 52,\n 99,\n 101,\n 48,\n 49,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64542 req_id = d043e99a-450a-406c-8c2a-65cd09fa964a
19442025-04-30T06:14:13.275ZDEBGtest_basic (store): query key key = c4e9f997-f271-4a58-92bc-01625e84ce01.host
19452025-04-30T06:14:13.276ZDEBGtest_basic (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:64542 query = LowerQuery { name: LowerName(Name("c4e9f997-f271-4a58-92bc-01625e84ce01.host.control-plane.oxide.internal.")), original: Query { name: Name("c4e9f997-f271-4a58-92bc-01625e84ce01.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("c4e9f997-f271-4a58-92bc-01625e84ce01.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = d043e99a-450a-406c-8c2a-65cd09fa964a
19462025-04-30T06:14:13.276ZINFOtest_basic (BackgroundTasks): found DNS servers (initial) addresses = DnsServersList { addresses: [[::1]:57818] } background_task = dns_servers_internal dns_group = internal
19472025-04-30T06:14:13.277ZDEBGtest_basic (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 42.998211ms iteration = 2 status = {"addresses":["[::1]:57818"]}
19482025-04-30T06:14:13.277ZINFOtest_basic (BackgroundTasks): found DNS servers (initial) addresses = DnsServersList { addresses: [[::1]:46146] } background_task = dns_servers_external dns_group = external
19492025-04-30T06:14:13.277ZDEBGtest_basic (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 42.304218ms iteration = 2 status = {"addresses":["[::1]:46146"]}
19502025-04-30T06:14:13.277ZDEBGtest_basic (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 2 reason = Dependency
19512025-04-30T06:14:13.277ZWARNtest_basic (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_internal dns_group = internal reason = no config
19522025-04-30T06:14:13.277ZDEBGtest_basic (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 40.962\u{b5}s iteration = 2 status = {"error":"no config"}
19532025-04-30T06:14:13.277ZDEBGtest_basic (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 2 reason = Dependency
19542025-04-30T06:14:13.278ZWARNtest_basic (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_external dns_group = external reason = no config
19552025-04-30T06:14:13.278ZDEBGtest_basic (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 10.271\u{b5}s iteration = 2 status = {"error":"no config"}
19562025-04-30T06:14:13.278ZDEBGtest_basic (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19572025-04-30T06:14:13.288ZDEBGtest_basic (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
19582025-04-30T06:14:13.288ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
19592025-04-30T06:14:13.288ZDEBGtest_basic (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
19602025-04-30T06:14:13.289ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
19612025-04-30T06:14:13.290ZDEBGtest_basic (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19622025-04-30T06:14:13.312ZDEBGtest_basic (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
19632025-04-30T06:14:13.317ZDEBGtest_basic (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19642025-04-30T06:14:13.333ZDEBGtest_basic (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: c19a698f-c6f9-4a17-ae30-20d711b8f7dc, lookup_type: ById(c19a698f-c6f9-4a17-ae30-20d711b8f7dc) } result = Ok(())
19652025-04-30T06:14:13.333ZINFOtest_basic (nexus): Rack initialized
19662025-04-30T06:14:13.333ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
19672025-04-30T06:14:13.335ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
19682025-04-30T06:14:13.337ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
19692025-04-30T06:14:13.337ZDEBGtest_basic (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19702025-04-30T06:14:13.376ZDEBGtest_basic (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
19712025-04-30T06:14:13.379ZTRACtest_basic (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig
19722025-04-30T06:14:13.379ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = external_endpoints resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19732025-04-30T06:14:13.380ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
19742025-04-30T06:14:13.380ZDEBGtest_basic (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
19752025-04-30T06:14:13.380ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
19762025-04-30T06:14:13.401ZDEBGtest_basic (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19772025-04-30T06:14:13.441ZDEBGtest_basic (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
19782025-04-30T06:14:13.441ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
19792025-04-30T06:14:13.441ZDEBGtest_basic (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
19802025-04-30T06:14:13.442ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
19812025-04-30T06:14:13.443ZDEBGtest_basic (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
19822025-04-30T06:14:13.443ZDEBGtest_basic (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
19832025-04-30T06:14:13.443ZDEBGtest_basic (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
19842025-04-30T06:14:13.443ZTRACtest_basic (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig
19852025-04-30T06:14:13.443ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_internal dns_group = internal resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19862025-04-30T06:14:13.443ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
19872025-04-30T06:14:13.443ZDEBGtest_basic (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
19882025-04-30T06:14:13.444ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
19892025-04-30T06:14:13.447ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
19902025-04-30T06:14:13.447ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
19912025-04-30T06:14:13.447ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
19922025-04-30T06:14:13.447ZDEBGtest_basic (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
19932025-04-30T06:14:13.447ZDEBGtest_basic (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
19942025-04-30T06:14:13.448ZDEBGtest_basic (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
19952025-04-30T06:14:13.448ZTRACtest_basic (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig
19962025-04-30T06:14:13.448ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_external dns_group = external resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19972025-04-30T06:14:13.448ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
19982025-04-30T06:14:13.448ZDEBGtest_basic (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
19992025-04-30T06:14:13.448ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
20002025-04-30T06:14:13.467ZDEBGtest_basic (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20012025-04-30T06:14:13.528ZDEBGtest_basic (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
20022025-04-30T06:14:13.528ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
20032025-04-30T06:14:13.528ZDEBGtest_basic (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
20042025-04-30T06:14:13.530ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
20052025-04-30T06:14:13.534ZDEBGtest_basic (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20062025-04-30T06:14:13.551ZDEBGtest_basic (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
20072025-04-30T06:14:13.556ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
20082025-04-30T06:14:13.557ZDEBGtest_basic (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
20092025-04-30T06:14:13.579ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
20102025-04-30T06:14:13.580ZTRACtest_basic (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet
20112025-04-30T06:14:13.581ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = external_endpoints resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
20122025-04-30T06:14:13.581ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
20132025-04-30T06:14:13.581ZDEBGtest_basic (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
20142025-04-30T06:14:13.588ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
20152025-04-30T06:14:13.592ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
20162025-04-30T06:14:13.592ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
20172025-04-30T06:14:13.599ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
20182025-04-30T06:14:13.627ZDEBGtest_basic (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
20192025-04-30T06:14:13.629ZDEBGtest_basic (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 8f651512-921c-4a36-b766-2ab13ca47a59 dns_zone_name = oxide-dev.test found_so_far = 0 version = 2
20202025-04-30T06:14:13.629ZTRACtest_basic (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig
20212025-04-30T06:14:13.636ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_external dns_group = external resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
20222025-04-30T06:14:13.636ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
20232025-04-30T06:14:13.638ZDEBGtest_basic (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
20242025-04-30T06:14:13.640ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
20252025-04-30T06:14:13.644ZDEBGtest_basic (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
20262025-04-30T06:14:13.644ZDEBGtest_basic (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 0e45000f-51b2-495f-a464-efdaeaf692c2 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
20272025-04-30T06:14:13.644ZTRACtest_basic (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig
20282025-04-30T06:14:13.644ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_internal dns_group = internal resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
20292025-04-30T06:14:13.644ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
20302025-04-30T06:14:13.645ZDEBGtest_basic (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
20312025-04-30T06:14:13.646ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
20322025-04-30T06:14:13.650ZDEBGtest_basic (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20332025-04-30T06:14:13.675ZDEBGtest_basic (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
20342025-04-30T06:14:13.676ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
20352025-04-30T06:14:13.676ZDEBGtest_basic (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
20362025-04-30T06:14:13.677ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
20372025-04-30T06:14:13.681ZDEBGtest_basic (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20382025-04-30T06:14:13.700ZDEBGtest_basic (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
20392025-04-30T06:14:13.700ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
20402025-04-30T06:14:13.700ZDEBGtest_basic (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
20412025-04-30T06:14:13.702ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
20422025-04-30T06:14:13.707ZDEBGtest_basic (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20432025-04-30T06:14:13.731ZDEBGtest_basic (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
20442025-04-30T06:14:13.731ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
20452025-04-30T06:14:13.731ZDEBGtest_basic (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
20462025-04-30T06:14:13.732ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
20472025-04-30T06:14:13.737ZDEBGtest_basic (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 8f651512-921c-4a36-b766-2ab13ca47a59 dns_zone_name = oxide-dev.test found_so_far = 1 version = 2
20482025-04-30T06:14:13.738ZDEBGtest_basic (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
20492025-04-30T06:14:13.738ZINFOtest_basic (BackgroundTasks): found latest generation (first find) background_task = dns_config_external dns_group = external generation = 2
20502025-04-30T06:14:13.738ZDEBGtest_basic (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 504.47279ms iteration = 2 status = {"generation":2}
20512025-04-30T06:14:13.739ZDEBGtest_basic (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 3 reason = Dependency
20522025-04-30T06:14:13.848ZDEBGtest_basic (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[::1]:46146 generation = 2 method = PUT servers = DnsServersList { addresses: [[::1]:46146] } uri = http://[::1]:46146/config
20532025-04-30T06:14:13.849ZDEBGtest_basic (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 0e45000f-51b2-495f-a464-efdaeaf692c2 dns_zone_name = control-plane.oxide.internal found_so_far = 21 version = 1
20542025-04-30T06:14:13.849ZDEBGtest_basic (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
20552025-04-30T06:14:13.849ZINFOtest_basic (BackgroundTasks): found latest generation (first find) background_task = dns_config_internal dns_group = internal generation = 1
20562025-04-30T06:14:13.849ZDEBGtest_basic (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 619.778047ms iteration = 2 status = {"generation":1}
20572025-04-30T06:14:13.849ZDEBGtest_basic (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 3 reason = Dependency
20582025-04-30T06:14:13.953ZDEBGtest_basic (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[::1]:57818 generation = 1 method = PUT servers = DnsServersList { addresses: [[::1]:57818] } uri = http://[::1]:57818/config
20592025-04-30T06:14:13.956ZINFOtest_basic (BackgroundTasks): found Silo/DNS/TLS config (initial) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide-dev.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-04-30T06:14:01.562204Z, time_modified: 2025-04-30T06:14:01.562204Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "test-suite-silo.sys.oxide-dev.test": ExternalEndpoint { silo_id: 96568163-7df0-4ae3-adb5-89dafc8f25cf, db_silo: Silo { identity: SiloIdentity { id: 96568163-7df0-4ae3-adb5-89dafc8f25cf, name: Name(Name("test-suite-silo")), description: "built-in recovery Silo", time_created: 2025-04-30T06:14:10.896091Z, time_modified: 2025-04-30T06:14:10.896091Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(1)) }, tls_certs: [] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide-dev.test" }, NoSiloCerts { silo_id: 96568163-7df0-4ae3-adb5-89dafc8f25cf, dns_name: "test-suite-silo.sys.oxide-dev.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 96568163-7df0-4ae3-adb5-89dafc8f25cf, db_silo: Silo { identity: SiloIdentity { id: 96568163-7df0-4ae3-adb5-89dafc8f25cf, name: Name(Name("test-suite-silo")), description: "built-in recovery Silo", time_created: 2025-04-30T06:14:10.896091Z, time_modified: 2025-04-30T06:14:10.896091Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(1)) }, tls_certs: [] }) }
20602025-04-30T06:14:13.957ZDEBGtest_basic (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 721.592306ms iteration = 2 status = {"by_dns_name":{"default-silo.sys.oxide-dev.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"test-suite-silo.sys.oxide-dev.test":{"silo_id":"96568163-7df0-4ae3-adb5-89dafc8f25cf","tls_certs":[]}},"default_endpoint":{"silo_id":"96568163-7df0-4ae3-adb5-89dafc8f25cf","tls_certs":[]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide-dev.test\\" has no usable certificates","silo 96568163-7df0-4ae3-adb5-89dafc8f25cf with DNS name \\"test-suite-silo.sys.oxide-dev.test\\" has no usable certificates"]}
20612025-04-30T06:14:13.959ZTRACtest_basic (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
20622025-04-30T06:14:13.960ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
20632025-04-30T06:14:13.960ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
20642025-04-30T06:14:13.960ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
20652025-04-30T06:14:13.961ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
20662025-04-30T06:14:13.962ZINFOtest_basic (http): accepted connection kind = dns local_addr = [::1]:57818 remote_addr = [::1]:48880
20672025-04-30T06:14:13.962ZINFOtest_basic (http): accepted connection kind = dns local_addr = [::1]:46146 remote_addr = [::1]:53722
20682025-04-30T06:14:13.965ZTRACtest_basic (http): incoming request kind = dns local_addr = [::1]:57818 method = PUT remote_addr = [::1]:48880 req_id = d4048225-75b0-4ed0-9869-e17b2d4ba6f7 uri = /config
20692025-04-30T06:14:13.965ZTRACtest_basic (http): incoming request kind = dns local_addr = [::1]:46146 method = PUT remote_addr = [::1]:53722 req_id = b8dd5276-ad9a-4d14-81bd-c3e4f554730d uri = /config
20702025-04-30T06:14:13.965ZINFOtest_basic (store): attempting generation update new_generation = 1 req_id = d4048225-75b0-4ed0-9869-e17b2d4ba6f7
20712025-04-30T06:14:13.965ZINFOtest_basic (store): updated generation new_generation = 1 req_id = d4048225-75b0-4ed0-9869-e17b2d4ba6f7
20722025-04-30T06:14:13.965ZINFOtest_basic (store): attempting generation update new_generation = 2 req_id = b8dd5276-ad9a-4d14-81bd-c3e4f554730d
20732025-04-30T06:14:13.965ZINFOtest_basic (store): pruning trees for generations newer than 0
20742025-04-30T06:14:13.965ZDEBGtest_basic (store): creating tree tree_name = generation_2_zone_oxide-dev.test
20752025-04-30T06:14:13.966ZDEBGtest_basic (store): updating current config
20762025-04-30T06:14:13.966ZDEBGtest_basic (store): flushing default tree
20772025-04-30T06:14:13.966ZINFOtest_basic (store): pruning trees for generations older than 2 keep = 3
20782025-04-30T06:14:13.966ZINFOtest_basic (store): updated generation new_generation = 2 req_id = b8dd5276-ad9a-4d14-81bd-c3e4f554730d
20792025-04-30T06:14:13.966ZINFOtest_basic (http): request completed kind = dns latency_us = 1487 local_addr = [::1]:57818 method = PUT remote_addr = [::1]:48880 req_id = d4048225-75b0-4ed0-9869-e17b2d4ba6f7 response_code = 204 uri = /config
20802025-04-30T06:14:13.966ZINFOtest_basic (http): request completed kind = dns latency_us = 1494 local_addr = [::1]:46146 method = PUT remote_addr = [::1]:53722 req_id = b8dd5276-ad9a-4d14-81bd-c3e4f554730d response_code = 204 uri = /config
20812025-04-30T06:14:13.966ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
20822025-04-30T06:14:13.966ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
20832025-04-30T06:14:13.966ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
20842025-04-30T06:14:13.967ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
20852025-04-30T06:14:13.968ZDEBGtest_basic (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[::1]:46146 generation = 2 result = Ok(Response { url: "http://[::1]:46146/config", status: 204, headers: {"x-request-id": "b8dd5276-ad9a-4d14-81bd-c3e4f554730d", "date": "Wed, 30 Apr 2025 06:14:13 GMT"} }) servers = DnsServersList { addresses: [[::1]:46146] }
20862025-04-30T06:14:13.968ZINFOtest_basic (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[::1]:46146 generation = 2 servers = DnsServersList { addresses: [[::1]:46146] }
20872025-04-30T06:14:13.970ZINFOtest_basic (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external generation = 2 servers = DnsServersList { addresses: [[::1]:46146] }
20882025-04-30T06:14:13.970ZDEBGtest_basic (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 231.693173ms iteration = 3 status = {"generation":2,"server_results":{"[::1]:46146":{"Ok":null}}}
20892025-04-30T06:14:13.971ZDEBGtest_basic (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[::1]:57818 generation = 1 result = Ok(Response { url: "http://[::1]:57818/config", status: 204, headers: {"x-request-id": "d4048225-75b0-4ed0-9869-e17b2d4ba6f7", "date": "Wed, 30 Apr 2025 06:14:13 GMT"} }) servers = DnsServersList { addresses: [[::1]:57818] }
20902025-04-30T06:14:13.971ZINFOtest_basic (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[::1]:57818 generation = 1 servers = DnsServersList { addresses: [[::1]:57818] }
20912025-04-30T06:14:13.973ZINFOtest_basic (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal generation = 1 servers = DnsServersList { addresses: [[::1]:57818] }
20922025-04-30T06:14:13.973ZDEBGtest_basic (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 123.682139ms iteration = 3 status = {"generation":1,"server_results":{"[::1]:57818":{"Ok":null}}}
20932025-04-30T06:14:13.978ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
20942025-04-30T06:14:13.978ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
20952025-04-30T06:14:13.978ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
20962025-04-30T06:14:13.988ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
20972025-04-30T06:14:13.997ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20982025-04-30T06:14:14.013ZDEBGtest_basic (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
20992025-04-30T06:14:14.013ZTRACtest_basic (InternalApi): successful lookup of siloed resource "Vpc" using built-in user
21002025-04-30T06:14:14.013ZTRACtest_basic (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
21012025-04-30T06:14:14.013ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
21022025-04-30T06:14:14.013ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21032025-04-30T06:14:14.013ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
21042025-04-30T06:14:14.013ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21052025-04-30T06:14:14.022ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
21062025-04-30T06:14:14.022ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21072025-04-30T06:14:14.023ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
21082025-04-30T06:14:14.025ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21092025-04-30T06:14:14.028ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
21102025-04-30T06:14:14.029ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21112025-04-30T06:14:14.029ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
21122025-04-30T06:14:14.030ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21132025-04-30T06:14:14.032ZINFOtest_basic (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = efcb0ee5-cafa-4692-9b75-fcee4732bdca
21142025-04-30T06:14:14.032ZDEBGtest_basic (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = efcb0ee5-cafa-4692-9b75-fcee4732bdca time_started = 2025-04-30 06:14:09.682169 UTC
21152025-04-30T06:14:14.033ZDEBGtest_basic (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 9.485532002s iteration = 2 status = {"collection_id":"efcb0ee5-cafa-4692-9b75-fcee4732bdca","time_done":"2025-04-30 06:14:12.846662 UTC","time_started":"2025-04-30 06:14:09.682169 UTC"}
21162025-04-30T06:14:14.033ZDEBGtest_basic (BackgroundTasks): activating background_task = inventory_collection iteration = 3 reason = Signaled
21172025-04-30T06:14:14.033ZTRACtest_basic (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
21182025-04-30T06:14:14.034ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
21192025-04-30T06:14:14.034ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
21202025-04-30T06:14:14.034ZDEBGtest_basic (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
21212025-04-30T06:14:14.035ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
21222025-04-30T06:14:14.035ZDEBGtest_basic (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 2 reason = Dependency
21232025-04-30T06:14:14.035ZDEBGtest_basic (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 210.91\u{b5}s iteration = 2 status = {"error":"task disabled"}
21242025-04-30T06:14:14.036ZDEBGtest_basic (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 2 reason = Dependency
21252025-04-30T06:14:14.036ZWARNtest_basic (BackgroundTasks): Blueprint rendezvous: skipped background_task = blueprint_rendezvous reason = no blueprint
21262025-04-30T06:14:14.036ZDEBGtest_basic (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 81.172\u{b5}s iteration = 2 status = {"error":"no blueprint"}
21272025-04-30T06:14:14.037ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21282025-04-30T06:14:14.052ZDEBGtest_basic (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
21292025-04-30T06:14:14.053ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21302025-04-30T06:14:14.053ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
21312025-04-30T06:14:14.055ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21322025-04-30T06:14:14.055ZDEBGtest_basic (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21332025-04-30T06:14:14.064ZDEBGtest_basic (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
21342025-04-30T06:14:14.064ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
21352025-04-30T06:14:14.064ZDEBGtest_basic (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
21362025-04-30T06:14:14.066ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
21372025-04-30T06:14:14.072ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21382025-04-30T06:14:14.072ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
21392025-04-30T06:14:14.075ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21402025-04-30T06:14:14.076ZDEBGtest_basic (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(efcb0ee5-cafa-4692-9b75-fcee4732bdca, 0)]
21412025-04-30T06:14:14.076ZTRACtest_basic (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
21422025-04-30T06:14:14.080ZTRACtest_basic (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: a0 ac 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:55605 req_id = c784bfda-94b4-4966-8e2a-120526763aa5
21432025-04-30T06:14:14.080ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 41132,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:55605 req_id = c784bfda-94b4-4966-8e2a-120526763aa5
21442025-04-30T06:14:14.081ZDEBGtest_basic (store): query key key = _mgs._tcp
21452025-04-30T06:14:14.081ZDEBGtest_basic (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
21462025-04-30T06:14:14.081ZDEBGtest_basic (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:55605 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 37822, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = c784bfda-94b4-4966-8e2a-120526763aa5
21472025-04-30T06:14:14.082ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21482025-04-30T06:14:14.082ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
21492025-04-30T06:14:14.083ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21502025-04-30T06:14:14.084ZDEBGtest_basic (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 37822, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2518, tv_nsec: 883014456 } })
21512025-04-30T06:14:14.084ZTRACtest_basic (DnsResolver): lookup_service_targets: looking up SRV target
21522025-04-30T06:14:14.089ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21532025-04-30T06:14:14.089ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
21542025-04-30T06:14:14.091ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21552025-04-30T06:14:14.094ZTRACtest_basic (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 97 4e 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .N..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:64337 req_id = 77fc3dbe-9c25-4ba6-81ef-bd5c99bd2be1
21562025-04-30T06:14:14.094ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 38734,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64337 req_id = 77fc3dbe-9c25-4ba6-81ef-bd5c99bd2be1
21572025-04-30T06:14:14.094ZDEBGtest_basic (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
21582025-04-30T06:14:14.094ZDEBGtest_basic (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:64337 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 77fc3dbe-9c25-4ba6-81ef-bd5c99bd2be1
21592025-04-30T06:14:14.176ZTRACtest_basic (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
21602025-04-30T06:14:14.177ZTRACtest_basic (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
21612025-04-30T06:14:14.177ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
21622025-04-30T06:14:14.177ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21632025-04-30T06:14:14.177ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
21642025-04-30T06:14:14.178ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21652025-04-30T06:14:14.182ZTRACtest_basic (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 67 b5 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c g............_cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:58505 req_id = 1e8b0c93-a6e6-4788-a63d-d14d53680ca5
21662025-04-30T06:14:14.182ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 26549,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:58505 req_id = 1e8b0c93-a6e6-4788-a63d-d14d53680ca5
21672025-04-30T06:14:14.183ZDEBGtest_basic (store): query key key = _clickhouse-admin-keeper._tcp
21682025-04-30T06:14:14.183ZERROtest_basic (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 26549, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:58505 req_id = 1e8b0c93-a6e6-4788-a63d-d14d53680ca5
21692025-04-30T06:14:14.184ZDEBGtest_basic (BackgroundTasks): begin collection background_task = inventory_collection
21702025-04-30T06:14:14.185ZDEBGtest_basic (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:37822
21712025-04-30T06:14:14.185ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/ignition
21722025-04-30T06:14:14.188ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
21732025-04-30T06:14:14.188ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21742025-04-30T06:14:14.188ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
21752025-04-30T06:14:14.190ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21762025-04-30T06:14:14.200ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
21772025-04-30T06:14:14.200ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21782025-04-30T06:14:14.200ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
21792025-04-30T06:14:14.200ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21802025-04-30T06:14:14.205ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "9ea12a2c-48f2-445f-aee8-760f49cdab4a", "content-length": "835", "date": "Wed, 30 Apr 2025 06:14:13 GMT"} })
21812025-04-30T06:14:14.205ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0
21822025-04-30T06:14:14.215ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21832025-04-30T06:14:14.228ZDEBGtest_basic (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
21842025-04-30T06:14:14.228ZTRACtest_basic (InternalApi): successful lookup of siloed resource "Vpc" using built-in user
21852025-04-30T06:14:14.228ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21862025-04-30T06:14:14.229ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
21872025-04-30T06:14:14.232ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21882025-04-30T06:14:14.256ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21892025-04-30T06:14:14.256ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
21902025-04-30T06:14:14.258ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21912025-04-30T06:14:14.258ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "795d682e-d06d-4e3b-85f1-6893671deb6f", "content-length": "734", "date": "Wed, 30 Apr 2025 06:14:13 GMT"} })
21922025-04-30T06:14:14.261ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=0
21932025-04-30T06:14:14.264ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21942025-04-30T06:14:14.265ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
21952025-04-30T06:14:14.265ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21962025-04-30T06:14:14.270ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21972025-04-30T06:14:14.270ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
21982025-04-30T06:14:14.273ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21992025-04-30T06:14:14.278ZTRACtest_basic (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") }
22002025-04-30T06:14:14.278ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
22012025-04-30T06:14:14.278ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22022025-04-30T06:14:14.278ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
22032025-04-30T06:14:14.279ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22042025-04-30T06:14:14.282ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4084082c-77e1-438d-b7c3-f10eefed80e3", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:13 GMT"} })
22052025-04-30T06:14:14.282ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=1
22062025-04-30T06:14:14.284ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
22072025-04-30T06:14:14.284ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22082025-04-30T06:14:14.284ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
22092025-04-30T06:14:14.287ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22102025-04-30T06:14:14.291ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
22112025-04-30T06:14:14.292ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22122025-04-30T06:14:14.293ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
22132025-04-30T06:14:14.294ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22142025-04-30T06:14:14.302ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22152025-04-30T06:14:14.326ZDEBGtest_basic (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
22162025-04-30T06:14:14.326ZTRACtest_basic (InternalApi): successful lookup of siloed resource "VpcSubnet" using built-in user
22172025-04-30T06:14:14.326ZTRACtest_basic (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") }
22182025-04-30T06:14:14.326ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
22192025-04-30T06:14:14.326ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22202025-04-30T06:14:14.326ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
22212025-04-30T06:14:14.327ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22222025-04-30T06:14:14.333ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "49df3ef0-78c0-48c9-a646-d523e421684c", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:13 GMT"} })
22232025-04-30T06:14:14.334ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=0
22242025-04-30T06:14:14.335ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
22252025-04-30T06:14:14.335ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22262025-04-30T06:14:14.335ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
22272025-04-30T06:14:14.338ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22282025-04-30T06:14:14.341ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
22292025-04-30T06:14:14.341ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22302025-04-30T06:14:14.341ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
22312025-04-30T06:14:14.342ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22322025-04-30T06:14:14.348ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22332025-04-30T06:14:14.366ZDEBGtest_basic (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
22342025-04-30T06:14:14.366ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22352025-04-30T06:14:14.366ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
22362025-04-30T06:14:14.369ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22372025-04-30T06:14:14.374ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "167f25b0-8673-4b32-93f5-b22e452443f9", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:13 GMT"} })
22382025-04-30T06:14:14.374ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=1
22392025-04-30T06:14:14.381ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22402025-04-30T06:14:14.381ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
22412025-04-30T06:14:14.383ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22422025-04-30T06:14:14.389ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22432025-04-30T06:14:14.389ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
22442025-04-30T06:14:14.390ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22452025-04-30T06:14:14.397ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22462025-04-30T06:14:14.397ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
22472025-04-30T06:14:14.398ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22482025-04-30T06:14:14.402ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1cccb05a-bd7f-4517-ae46-9e04aa36b345", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:13 GMT"} })
22492025-04-30T06:14:14.412ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=0
22502025-04-30T06:14:14.421ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22512025-04-30T06:14:14.421ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
22522025-04-30T06:14:14.439ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22532025-04-30T06:14:14.453ZTRACtest_basic (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") }
22542025-04-30T06:14:14.454ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
22552025-04-30T06:14:14.454ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22562025-04-30T06:14:14.454ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
22572025-04-30T06:14:14.457ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22582025-04-30T06:14:14.470ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
22592025-04-30T06:14:14.471ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22602025-04-30T06:14:14.471ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
22612025-04-30T06:14:14.475ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22622025-04-30T06:14:14.479ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
22632025-04-30T06:14:14.479ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22642025-04-30T06:14:14.480ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
22652025-04-30T06:14:14.488ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22662025-04-30T06:14:14.494ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c021050d-e8b4-445b-a614-eec595a142c3", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:13 GMT"} })
22672025-04-30T06:14:14.494ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=1
22682025-04-30T06:14:14.498ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22692025-04-30T06:14:14.537ZDEBGtest_basic (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
22702025-04-30T06:14:14.537ZTRACtest_basic (InternalApi): successful lookup of siloed resource "VpcSubnet" using built-in user
22712025-04-30T06:14:14.537ZTRACtest_basic (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") }
22722025-04-30T06:14:14.537ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
22732025-04-30T06:14:14.538ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22742025-04-30T06:14:14.538ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
22752025-04-30T06:14:14.541ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22762025-04-30T06:14:14.546ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
22772025-04-30T06:14:14.546ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22782025-04-30T06:14:14.546ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
22792025-04-30T06:14:14.549ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22802025-04-30T06:14:14.553ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
22812025-04-30T06:14:14.553ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22822025-04-30T06:14:14.554ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
22832025-04-30T06:14:14.558ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22842025-04-30T06:14:14.566ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22852025-04-30T06:14:14.582ZDEBGtest_basic (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
22862025-04-30T06:14:14.583ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22872025-04-30T06:14:14.583ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
22882025-04-30T06:14:14.587ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22892025-04-30T06:14:14.599ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ae470764-a8e1-49d6-9188-03c84d323d6b", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:13 GMT"} })
22902025-04-30T06:14:14.600ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cmpa
22912025-04-30T06:14:14.610ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b7dad6c5-cadb-4b77-a48b-81146ddd58bd", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:13 GMT"} })
22922025-04-30T06:14:14.611ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
22932025-04-30T06:14:14.612ZDEBGtest_basic (InternalApi): resolved names for firewall rules instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 53d2e1ed-df5d-4f37-aa3e-e472d5878035, kind: Service { id: 088ac96c-981b-4054-bea6-d8ab23ae1366 }, name: Name("nexus-088ac96c-981b-4054-bea6-d8ab23ae1366"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 1])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("external-dns"): [NetworkInterface { id: a5b14f67-66ad-4f05-bc8e-859cf15ab7f4, kind: Service { id: c4e9f997-f271-4a58-92bc-01625e84ce01 }, name: Name("external-dns-c4e9f997-f271-4a58-92bc-01625e84ce01"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 0])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
22942025-04-30T06:14:14.612ZTRACtest_basic (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
22952025-04-30T06:14:14.612ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
22962025-04-30T06:14:14.612ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22972025-04-30T06:14:14.612ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
22982025-04-30T06:14:14.613ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22992025-04-30T06:14:14.617ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6eb4987f-21a8-404b-837b-9334ac3a8d8c", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:13 GMT"} })
23002025-04-30T06:14:14.618ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
23012025-04-30T06:14:14.618ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23022025-04-30T06:14:14.636ZDEBGtest_basic (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
23032025-04-30T06:14:14.636ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23042025-04-30T06:14:14.636ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
23052025-04-30T06:14:14.638ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23062025-04-30T06:14:14.644ZTRACtest_basic (InternalApi): fetched allowlist from DB allowed = AllowList { id: 001de000-a110-4000-8000-000000000000, time_created: 2025-04-30T06:06:08.248500Z, time_modified: 2025-04-30T06:14:11.128781Z, allowed_ips: None }
23072025-04-30T06:14:14.644ZDEBGtest_basic (InternalApi): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified.
23082025-04-30T06:14:14.645ZDEBGtest_basic (InternalApi): resolved firewall rules for sled agents sled_agent_rules = [ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: a5b14f67-66ad-4f05-bc8e-859cf15ab7f4, kind: Service { id: c4e9f997-f271-4a58-92bc-01625e84ce01 }, name: Name("external-dns-c4e9f997-f271-4a58-92bc-01625e84ce01"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 0])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }, ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 53d2e1ed-df5d-4f37-aa3e-e472d5878035, kind: Service { id: 088ac96c-981b-4054-bea6-d8ab23ae1366 }, name: Name("nexus-088ac96c-981b-4054-bea6-d8ab23ae1366"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 1])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }]
23092025-04-30T06:14:14.645ZDEBGtest_basic (InternalApi): resolved 2 rules for sleds
23102025-04-30T06:14:14.649ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "faade54d-ff6e-4c30-99f7-072f04d37e07", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:13 GMT"} })
23112025-04-30T06:14:14.651ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
23122025-04-30T06:14:14.666ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "34bab443-fefe-4d79-aa2f-f78b756068e1", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:13 GMT"} })
23132025-04-30T06:14:14.666ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1
23142025-04-30T06:14:14.672ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4f12d7ba-35fd-499b-8c92-0e29bd48e424", "content-length": "734", "date": "Wed, 30 Apr 2025 06:14:13 GMT"} })
23152025-04-30T06:14:14.672ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=0
23162025-04-30T06:14:14.682ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "136b3fb2-0291-4ec9-aa67-abdc11cdcd2f", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:13 GMT"} })
23172025-04-30T06:14:14.682ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=1
23182025-04-30T06:14:14.687ZDEBGtest_basic (InternalApi): resolved sleds for vpc oxide-services vpc_to_sled = [Sled { identity: SledIdentity { id: b6d65341-167c-41df-9b5c-41cded99c229, time_created: 2025-04-30T06:14:03.570056Z, time_modified: 2025-04-30T06:14:03.570056Z }, time_deleted: None, rcgen: Generation(Generation(1)), rack_id: c19a698f-c6f9-4a17-ae30-20d711b8f7dc, is_scrimlet: true, serial_number: "sim-b6d65341", part_number: "Unknown", revision: SqlU32(0), usable_hardware_threads: SqlU32(16), usable_physical_ram: ByteCount(ByteCount(34359738368)), reservoir_size: ByteCount(ByteCount(17179869184)), ip: ::1, port: SqlU16(42420), last_used_address: ::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(51632) }]
23192025-04-30T06:14:14.687ZDEBGtest_basic (InternalApi): sending firewall rules to sled agents
23202025-04-30T06:14:14.687ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23212025-04-30T06:14:14.687ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
23222025-04-30T06:14:14.688ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23232025-04-30T06:14:14.689ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "dfcb8a62-7f4f-4703-af3d-383053056dd0", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:13 GMT"} })
23242025-04-30T06:14:14.689ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=0
23252025-04-30T06:14:14.695ZTRACtest_basic (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Sled { parent: Fleet, key: b6d65341-167c-41df-9b5c-41cded99c229, lookup_type: ById(b6d65341-167c-41df-9b5c-41cded99c229) }
23262025-04-30T06:14:14.696ZTRACtest_basic (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
23272025-04-30T06:14:14.696ZTRACtest_basic (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23282025-04-30T06:14:14.696ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {} }
23292025-04-30T06:14:14.699ZDEBGtest_basic (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23302025-04-30T06:14:14.704ZDEBGtest_basic (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23312025-04-30T06:14:14.728ZDEBGtest_basic (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Sled { parent: Fleet, key: b6d65341-167c-41df-9b5c-41cded99c229, lookup_type: ById(b6d65341-167c-41df-9b5c-41cded99c229) } result = Ok(())
23322025-04-30T06:14:14.835ZDEBGtest_basic (InternalApi): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 body = Some(Body) method = PUT uri = http://[::1]:42420/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
23332025-04-30T06:14:14.837ZINFOtest_basic (dropshot): accepted connection local_addr = [::1]:42420 remote_addr = [::1]:47851 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
23342025-04-30T06:14:14.840ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "05de66de-dfc8-4498-8cc5-056749ee2d5d", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:13 GMT"} })
23352025-04-30T06:14:14.841ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=1
23362025-04-30T06:14:14.844ZTRACtest_basic (dropshot): incoming request local_addr = [::1]:42420 method = PUT remote_addr = [::1]:47851 req_id = b8fbd250-471e-4275-8f3b-36bea36fb54d sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
23372025-04-30T06:14:14.848ZINFOtest_basic (dropshot): request completed latency_us = 3476 local_addr = [::1]:42420 method = PUT remote_addr = [::1]:47851 req_id = b8fbd250-471e-4275-8f3b-36bea36fb54d response_code = 204 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
23382025-04-30T06:14:14.849ZDEBGtest_basic (InternalApi): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 result = Ok(Response { url: "http://[::1]:42420/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "b8fbd250-471e-4275-8f3b-36bea36fb54d", "date": "Wed, 30 Apr 2025 06:14:13 GMT"} })
23392025-04-30T06:14:14.852ZINFOtest_basic (InternalApi): updated firewall rules on 1 sleds
23402025-04-30T06:14:14.852ZINFOtest_basic (nexus): plumbed initial IP allowlist
23412025-04-30T06:14:14.961ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = / versions = all
23422025-04-30T06:14:14.961ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /access versions = all
23432025-04-30T06:14:14.961ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /assets/{path} versions = all
23442025-04-30T06:14:14.961ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /device/auth versions = all
23452025-04-30T06:14:14.961ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /device/confirm versions = all
23462025-04-30T06:14:14.961ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /device/success versions = all
23472025-04-30T06:14:14.961ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /device/token versions = all
23482025-04-30T06:14:14.961ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /device/verify versions = all
23492025-04-30T06:14:14.961ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /experimental/v1/probes versions = all
23502025-04-30T06:14:14.961ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /experimental/v1/probes versions = all
23512025-04-30T06:14:14.961ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = DELETE path = /experimental/v1/probes/{probe} versions = all
23522025-04-30T06:14:14.961ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /experimental/v1/probes/{probe} versions = all
23532025-04-30T06:14:14.961ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /experimental/v1/system/support-bundles versions = all
23542025-04-30T06:14:14.961ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /experimental/v1/system/support-bundles versions = all
23552025-04-30T06:14:14.961ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = DELETE path = /experimental/v1/system/support-bundles/{support_bundle} versions = all
23562025-04-30T06:14:14.962ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /experimental/v1/system/support-bundles/{support_bundle} versions = all
23572025-04-30T06:14:14.962ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /experimental/v1/system/support-bundles/{support_bundle}/download versions = all
23582025-04-30T06:14:14.962ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = HEAD path = /experimental/v1/system/support-bundles/{support_bundle}/download versions = all
23592025-04-30T06:14:14.962ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /experimental/v1/system/support-bundles/{support_bundle}/download/{file} versions = all
23602025-04-30T06:14:14.962ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = HEAD path = /experimental/v1/system/support-bundles/{support_bundle}/download/{file} versions = all
23612025-04-30T06:14:14.962ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /experimental/v1/system/support-bundles/{support_bundle}/index versions = all
23622025-04-30T06:14:14.962ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /images versions = all
23632025-04-30T06:14:14.962ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /login versions = all
23642025-04-30T06:14:14.962ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /login/{silo_name}/local versions = all
23652025-04-30T06:14:14.962ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /login/{silo_name}/saml/{provider_name} versions = all
23662025-04-30T06:14:14.962ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /login/{silo_name}/saml/{provider_name} versions = all
23672025-04-30T06:14:14.962ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /login/{silo_name}/saml/{provider_name}/redirect versions = all
23682025-04-30T06:14:14.962ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /lookup/{path} versions = all
23692025-04-30T06:14:14.962ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /projects/{path} versions = all
23702025-04-30T06:14:14.962ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /projects-new versions = all
23712025-04-30T06:14:14.962ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /settings/{path} versions = all
23722025-04-30T06:14:14.962ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /system/{path} versions = all
23732025-04-30T06:14:14.962ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /utilization versions = all
23742025-04-30T06:14:14.962ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/affinity-groups versions = all
23752025-04-30T06:14:14.962ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/affinity-groups versions = all
23762025-04-30T06:14:14.962ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = DELETE path = /v1/affinity-groups/{affinity_group} versions = all
23772025-04-30T06:14:14.962ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/affinity-groups/{affinity_group} versions = all
23782025-04-30T06:14:14.962ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = PUT path = /v1/affinity-groups/{affinity_group} versions = all
23792025-04-30T06:14:14.962ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/affinity-groups/{affinity_group}/members versions = all
23802025-04-30T06:14:14.962ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = DELETE path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
23812025-04-30T06:14:14.962ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
23822025-04-30T06:14:14.962ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
23832025-04-30T06:14:14.962ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/anti-affinity-groups versions = all
23842025-04-30T06:14:14.962ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/anti-affinity-groups versions = all
23852025-04-30T06:14:14.962ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = DELETE path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
23862025-04-30T06:14:14.962ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
23872025-04-30T06:14:14.962ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = PUT path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
23882025-04-30T06:14:14.962ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group}/members versions = all
23892025-04-30T06:14:14.962ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = DELETE path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
23902025-04-30T06:14:14.962ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
23912025-04-30T06:14:14.962ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
23922025-04-30T06:14:14.962ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/certificates versions = all
23932025-04-30T06:14:14.962ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/certificates versions = all
23942025-04-30T06:14:14.962ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = DELETE path = /v1/certificates/{certificate} versions = all
23952025-04-30T06:14:14.962ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/certificates/{certificate} versions = all
23962025-04-30T06:14:14.962ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/disks versions = all
23972025-04-30T06:14:14.962ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/disks versions = all
23982025-04-30T06:14:14.962ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = DELETE path = /v1/disks/{disk} versions = all
23992025-04-30T06:14:14.962ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/disks/{disk} versions = all
24002025-04-30T06:14:14.963ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/disks/{disk}/bulk-write versions = all
24012025-04-30T06:14:14.963ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/disks/{disk}/bulk-write-start versions = all
24022025-04-30T06:14:14.963ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/disks/{disk}/bulk-write-stop versions = all
24032025-04-30T06:14:14.963ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/disks/{disk}/finalize versions = all
24042025-04-30T06:14:14.963ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/disks/{disk}/metrics/{metric} versions = all
24052025-04-30T06:14:14.963ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/floating-ips versions = all
24062025-04-30T06:14:14.963ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/floating-ips versions = all
24072025-04-30T06:14:14.963ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = DELETE path = /v1/floating-ips/{floating_ip} versions = all
24082025-04-30T06:14:14.963ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/floating-ips/{floating_ip} versions = all
24092025-04-30T06:14:14.963ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = PUT path = /v1/floating-ips/{floating_ip} versions = all
24102025-04-30T06:14:14.963ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/floating-ips/{floating_ip}/attach versions = all
24112025-04-30T06:14:14.963ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/floating-ips/{floating_ip}/detach versions = all
24122025-04-30T06:14:14.963ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/groups versions = all
24132025-04-30T06:14:14.963ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/groups/{group_id} versions = all
24142025-04-30T06:14:14.963ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/images versions = all
24152025-04-30T06:14:14.963ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/images versions = all
24162025-04-30T06:14:14.963ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = DELETE path = /v1/images/{image} versions = all
24172025-04-30T06:14:14.963ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/images/{image} versions = all
24182025-04-30T06:14:14.963ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/images/{image}/demote versions = all
24192025-04-30T06:14:14.963ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/images/{image}/promote versions = all
24202025-04-30T06:14:14.963ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/instances versions = all
24212025-04-30T06:14:14.963ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/instances versions = all
24222025-04-30T06:14:14.963ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = DELETE path = /v1/instances/{instance} versions = all
24232025-04-30T06:14:14.963ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/instances/{instance} versions = all
24242025-04-30T06:14:14.963ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = PUT path = /v1/instances/{instance} versions = all
24252025-04-30T06:14:14.963ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/instances/{instance}/affinity-groups versions = all
24262025-04-30T06:14:14.963ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/instances/{instance}/anti-affinity-groups versions = all
24272025-04-30T06:14:14.963ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/instances/{instance}/disks versions = all
24282025-04-30T06:14:14.963ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/instances/{instance}/disks/attach versions = all
24292025-04-30T06:14:14.963ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/instances/{instance}/disks/detach versions = all
24302025-04-30T06:14:14.963ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/instances/{instance}/external-ips versions = all
24312025-04-30T06:14:14.963ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = DELETE path = /v1/instances/{instance}/external-ips/ephemeral versions = all
24322025-04-30T06:14:14.963ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/instances/{instance}/external-ips/ephemeral versions = all
24332025-04-30T06:14:14.963ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/instances/{instance}/reboot versions = all
24342025-04-30T06:14:14.963ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/instances/{instance}/serial-console versions = all
24352025-04-30T06:14:14.963ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/instances/{instance}/serial-console/stream versions = all
24362025-04-30T06:14:14.963ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/instances/{instance}/ssh-public-keys versions = all
24372025-04-30T06:14:14.963ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/instances/{instance}/start versions = all
24382025-04-30T06:14:14.963ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/instances/{instance}/stop versions = all
24392025-04-30T06:14:14.963ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/internet-gateway-ip-addresses versions = all
24402025-04-30T06:14:14.963ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/internet-gateway-ip-addresses versions = all
24412025-04-30T06:14:14.963ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = DELETE path = /v1/internet-gateway-ip-addresses/{address} versions = all
24422025-04-30T06:14:14.963ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/internet-gateway-ip-pools versions = all
24432025-04-30T06:14:14.963ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/internet-gateway-ip-pools versions = all
24442025-04-30T06:14:14.963ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = DELETE path = /v1/internet-gateway-ip-pools/{pool} versions = all
24452025-04-30T06:14:14.963ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/internet-gateways versions = all
24462025-04-30T06:14:14.964ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/internet-gateways versions = all
24472025-04-30T06:14:14.964ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = DELETE path = /v1/internet-gateways/{gateway} versions = all
24482025-04-30T06:14:14.964ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/internet-gateways/{gateway} versions = all
24492025-04-30T06:14:14.964ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/ip-pools versions = all
24502025-04-30T06:14:14.964ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/ip-pools/{pool} versions = all
24512025-04-30T06:14:14.964ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/login/{silo_name}/local versions = all
24522025-04-30T06:14:14.964ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/logout versions = all
24532025-04-30T06:14:14.964ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/me versions = all
24542025-04-30T06:14:14.964ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/me/groups versions = all
24552025-04-30T06:14:14.964ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/me/ssh-keys versions = all
24562025-04-30T06:14:14.964ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/me/ssh-keys versions = all
24572025-04-30T06:14:14.964ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = DELETE path = /v1/me/ssh-keys/{ssh_key} versions = all
24582025-04-30T06:14:14.964ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/me/ssh-keys/{ssh_key} versions = all
24592025-04-30T06:14:14.964ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/metrics/{metric_name} versions = all
24602025-04-30T06:14:14.964ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/network-interfaces versions = all
24612025-04-30T06:14:14.964ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/network-interfaces versions = all
24622025-04-30T06:14:14.964ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = DELETE path = /v1/network-interfaces/{interface} versions = all
24632025-04-30T06:14:14.964ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/network-interfaces/{interface} versions = all
24642025-04-30T06:14:14.964ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = PUT path = /v1/network-interfaces/{interface} versions = all
24652025-04-30T06:14:14.964ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/ping versions = all
24662025-04-30T06:14:14.964ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/policy versions = all
24672025-04-30T06:14:14.964ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = PUT path = /v1/policy versions = all
24682025-04-30T06:14:14.964ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/projects versions = all
24692025-04-30T06:14:14.964ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/projects versions = all
24702025-04-30T06:14:14.964ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = DELETE path = /v1/projects/{project} versions = all
24712025-04-30T06:14:14.964ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/projects/{project} versions = all
24722025-04-30T06:14:14.964ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = PUT path = /v1/projects/{project} versions = all
24732025-04-30T06:14:14.964ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/projects/{project}/policy versions = all
24742025-04-30T06:14:14.964ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = PUT path = /v1/projects/{project}/policy versions = all
24752025-04-30T06:14:14.964ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/snapshots versions = all
24762025-04-30T06:14:14.964ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/snapshots versions = all
24772025-04-30T06:14:14.964ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = DELETE path = /v1/snapshots/{snapshot} versions = all
24782025-04-30T06:14:14.964ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/snapshots/{snapshot} versions = all
24792025-04-30T06:14:14.964ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/hardware/disks versions = all
24802025-04-30T06:14:14.964ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/hardware/disks/{disk_id} versions = all
24812025-04-30T06:14:14.964ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/hardware/rack-switch-port/{rack_id}/{switch_location}/{port}/lldp/neighbors versions = all
24822025-04-30T06:14:14.964ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/hardware/racks versions = all
24832025-04-30T06:14:14.964ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/hardware/racks/{rack_id} versions = all
24842025-04-30T06:14:14.964ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/hardware/sleds versions = all
24852025-04-30T06:14:14.964ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/system/hardware/sleds versions = all
24862025-04-30T06:14:14.964ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/hardware/sleds/{sled_id} versions = all
24872025-04-30T06:14:14.964ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/hardware/sleds/{sled_id}/disks versions = all
24882025-04-30T06:14:14.964ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/hardware/sleds/{sled_id}/instances versions = all
24892025-04-30T06:14:14.964ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = PUT path = /v1/system/hardware/sleds/{sled_id}/provision-policy versions = all
24902025-04-30T06:14:14.965ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/hardware/sleds-uninitialized versions = all
24912025-04-30T06:14:14.965ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/hardware/switch-port versions = all
24922025-04-30T06:14:14.965ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/hardware/switch-port/{port}/lldp/config versions = all
24932025-04-30T06:14:14.965ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/system/hardware/switch-port/{port}/lldp/config versions = all
24942025-04-30T06:14:14.965ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = DELETE path = /v1/system/hardware/switch-port/{port}/settings versions = all
24952025-04-30T06:14:14.965ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/system/hardware/switch-port/{port}/settings versions = all
24962025-04-30T06:14:14.965ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/hardware/switch-port/{port}/status versions = all
24972025-04-30T06:14:14.965ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/hardware/switches versions = all
24982025-04-30T06:14:14.965ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/hardware/switches/{switch_id} versions = all
24992025-04-30T06:14:14.965ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/identity-providers versions = all
25002025-04-30T06:14:14.965ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/system/identity-providers/local/users versions = all
25012025-04-30T06:14:14.965ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = DELETE path = /v1/system/identity-providers/local/users/{user_id} versions = all
25022025-04-30T06:14:14.965ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/system/identity-providers/local/users/{user_id}/set-password versions = all
25032025-04-30T06:14:14.965ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/system/identity-providers/saml versions = all
25042025-04-30T06:14:14.965ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/identity-providers/saml/{provider} versions = all
25052025-04-30T06:14:14.965ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/ip-pools versions = all
25062025-04-30T06:14:14.965ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/system/ip-pools versions = all
25072025-04-30T06:14:14.965ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = DELETE path = /v1/system/ip-pools/{pool} versions = all
25082025-04-30T06:14:14.965ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/ip-pools/{pool} versions = all
25092025-04-30T06:14:14.965ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = PUT path = /v1/system/ip-pools/{pool} versions = all
25102025-04-30T06:14:14.965ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/ip-pools/{pool}/ranges versions = all
25112025-04-30T06:14:14.965ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/system/ip-pools/{pool}/ranges/add versions = all
25122025-04-30T06:14:14.965ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/system/ip-pools/{pool}/ranges/remove versions = all
25132025-04-30T06:14:14.965ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/ip-pools/{pool}/silos versions = all
25142025-04-30T06:14:14.965ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/system/ip-pools/{pool}/silos versions = all
25152025-04-30T06:14:14.965ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = DELETE path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
25162025-04-30T06:14:14.965ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = PUT path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
25172025-04-30T06:14:14.965ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/ip-pools/{pool}/utilization versions = all
25182025-04-30T06:14:14.965ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/ip-pools-service versions = all
25192025-04-30T06:14:14.965ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/ip-pools-service/ranges versions = all
25202025-04-30T06:14:14.965ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/system/ip-pools-service/ranges/add versions = all
25212025-04-30T06:14:14.965ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/system/ip-pools-service/ranges/remove versions = all
25222025-04-30T06:14:14.965ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/metrics/{metric_name} versions = all
25232025-04-30T06:14:14.965ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/networking/address-lot versions = all
25242025-04-30T06:14:14.965ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/system/networking/address-lot versions = all
25252025-04-30T06:14:14.965ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = DELETE path = /v1/system/networking/address-lot/{address_lot} versions = all
25262025-04-30T06:14:14.966ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/networking/address-lot/{address_lot}/blocks versions = all
25272025-04-30T06:14:14.966ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/networking/allow-list versions = all
25282025-04-30T06:14:14.966ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = PUT path = /v1/system/networking/allow-list versions = all
25292025-04-30T06:14:14.966ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/system/networking/bfd-disable versions = all
25302025-04-30T06:14:14.966ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/system/networking/bfd-enable versions = all
25312025-04-30T06:14:14.966ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/networking/bfd-status versions = all
25322025-04-30T06:14:14.966ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = DELETE path = /v1/system/networking/bgp versions = all
25332025-04-30T06:14:14.966ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/networking/bgp versions = all
25342025-04-30T06:14:14.966ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/system/networking/bgp versions = all
25352025-04-30T06:14:14.966ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/networking/bgp-announce-set versions = all
25362025-04-30T06:14:14.966ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = PUT path = /v1/system/networking/bgp-announce-set versions = all
25372025-04-30T06:14:14.966ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = DELETE path = /v1/system/networking/bgp-announce-set/{announce_set} versions = all
25382025-04-30T06:14:14.966ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/networking/bgp-announce-set/{announce_set}/announcement versions = all
25392025-04-30T06:14:14.966ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/networking/bgp-exported versions = all
25402025-04-30T06:14:14.966ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/networking/bgp-message-history versions = all
25412025-04-30T06:14:14.966ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/networking/bgp-routes-ipv4 versions = all
25422025-04-30T06:14:14.966ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/networking/bgp-status versions = all
25432025-04-30T06:14:14.966ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/networking/loopback-address versions = all
25442025-04-30T06:14:14.966ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/system/networking/loopback-address versions = all
25452025-04-30T06:14:14.966ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = DELETE path = /v1/system/networking/loopback-address/{rack_id}/{switch_location}/{address}/{subnet_mask} versions = all
25462025-04-30T06:14:14.966ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = DELETE path = /v1/system/networking/switch-port-settings versions = all
25472025-04-30T06:14:14.966ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/networking/switch-port-settings versions = all
25482025-04-30T06:14:14.966ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/system/networking/switch-port-settings versions = all
25492025-04-30T06:14:14.966ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/networking/switch-port-settings/{port} versions = all
25502025-04-30T06:14:14.966ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/policy versions = all
25512025-04-30T06:14:14.966ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = PUT path = /v1/system/policy versions = all
25522025-04-30T06:14:14.966ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/roles versions = all
25532025-04-30T06:14:14.966ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/roles/{role_name} versions = all
25542025-04-30T06:14:14.966ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/silo-quotas versions = all
25552025-04-30T06:14:14.966ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/silos versions = all
25562025-04-30T06:14:14.966ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/system/silos versions = all
25572025-04-30T06:14:14.966ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = DELETE path = /v1/system/silos/{silo} versions = all
25582025-04-30T06:14:14.966ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/silos/{silo} versions = all
25592025-04-30T06:14:14.966ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/silos/{silo}/ip-pools versions = all
25602025-04-30T06:14:14.966ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/silos/{silo}/policy versions = all
25612025-04-30T06:14:14.966ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = PUT path = /v1/system/silos/{silo}/policy versions = all
25622025-04-30T06:14:14.966ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/silos/{silo}/quotas versions = all
25632025-04-30T06:14:14.966ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = PUT path = /v1/system/silos/{silo}/quotas versions = all
25642025-04-30T06:14:14.966ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/system/timeseries/query versions = all
25652025-04-30T06:14:14.966ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/timeseries/schemas versions = all
25662025-04-30T06:14:14.966ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = PUT path = /v1/system/update/repository versions = all
25672025-04-30T06:14:14.966ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/update/repository/{system_version} versions = all
25682025-04-30T06:14:14.967ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/update/target-release versions = all
25692025-04-30T06:14:14.967ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = PUT path = /v1/system/update/target-release versions = all
25702025-04-30T06:14:14.967ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/users versions = all
25712025-04-30T06:14:14.967ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/users/{user_id} versions = all
25722025-04-30T06:14:14.967ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/users-builtin versions = all
25732025-04-30T06:14:14.967ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/users-builtin/{user} versions = all
25742025-04-30T06:14:14.967ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/utilization/silos versions = all
25752025-04-30T06:14:14.967ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/system/utilization/silos/{silo} versions = all
25762025-04-30T06:14:14.967ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/timeseries/query versions = all
25772025-04-30T06:14:14.967ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/users versions = all
25782025-04-30T06:14:14.967ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/utilization versions = all
25792025-04-30T06:14:14.967ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/vpc-firewall-rules versions = all
25802025-04-30T06:14:14.967ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = PUT path = /v1/vpc-firewall-rules versions = all
25812025-04-30T06:14:14.967ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/vpc-router-routes versions = all
25822025-04-30T06:14:14.967ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/vpc-router-routes versions = all
25832025-04-30T06:14:14.967ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = DELETE path = /v1/vpc-router-routes/{route} versions = all
25842025-04-30T06:14:14.967ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/vpc-router-routes/{route} versions = all
25852025-04-30T06:14:14.967ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = PUT path = /v1/vpc-router-routes/{route} versions = all
25862025-04-30T06:14:14.967ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/vpc-routers versions = all
25872025-04-30T06:14:14.967ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/vpc-routers versions = all
25882025-04-30T06:14:14.967ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = DELETE path = /v1/vpc-routers/{router} versions = all
25892025-04-30T06:14:14.967ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/vpc-routers/{router} versions = all
25902025-04-30T06:14:14.967ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = PUT path = /v1/vpc-routers/{router} versions = all
25912025-04-30T06:14:14.967ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/vpc-subnets versions = all
25922025-04-30T06:14:14.967ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/vpc-subnets versions = all
25932025-04-30T06:14:14.967ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = DELETE path = /v1/vpc-subnets/{subnet} versions = all
25942025-04-30T06:14:14.967ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/vpc-subnets/{subnet} versions = all
25952025-04-30T06:14:14.967ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = PUT path = /v1/vpc-subnets/{subnet} versions = all
25962025-04-30T06:14:14.967ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/vpc-subnets/{subnet}/network-interfaces versions = all
25972025-04-30T06:14:14.967ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/vpcs versions = all
25982025-04-30T06:14:14.967ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/vpcs versions = all
25992025-04-30T06:14:14.967ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = DELETE path = /v1/vpcs/{vpc} versions = all
26002025-04-30T06:14:14.967ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/vpcs/{vpc} versions = all
26012025-04-30T06:14:14.967ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = PUT path = /v1/vpcs/{vpc} versions = all
26022025-04-30T06:14:14.967ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/webhooks/deliveries versions = all
26032025-04-30T06:14:14.967ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/webhooks/deliveries/{event_id}/resend versions = all
26042025-04-30T06:14:14.967ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/webhooks/event-classes versions = all
26052025-04-30T06:14:14.967ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/webhooks/receivers versions = all
26062025-04-30T06:14:14.967ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/webhooks/receivers versions = all
26072025-04-30T06:14:14.967ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = DELETE path = /v1/webhooks/receivers/{receiver} versions = all
26082025-04-30T06:14:14.967ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/webhooks/receivers/{receiver} versions = all
26092025-04-30T06:14:14.967ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = PUT path = /v1/webhooks/receivers/{receiver} versions = all
26102025-04-30T06:14:14.968ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/webhooks/receivers/{receiver}/probe versions = all
26112025-04-30T06:14:14.968ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/webhooks/receivers/{receiver}/subscriptions versions = all
26122025-04-30T06:14:14.968ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = DELETE path = /v1/webhooks/receivers/{receiver}/subscriptions/{subscription} versions = all
26132025-04-30T06:14:14.968ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = GET path = /v1/webhooks/secrets versions = all
26142025-04-30T06:14:14.968ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = POST path = /v1/webhooks/secrets versions = all
26152025-04-30T06:14:14.968ZDEBGtest_basic (dropshot_external): registered endpoint local_addr = 127.0.0.1:49597 method = DELETE path = /v1/webhooks/secrets/{secret_id} versions = all
26162025-04-30T06:14:14.968ZINFOtest_basic (dropshot_external): listening local_addr = 127.0.0.1:49597
26172025-04-30T06:14:14.968ZDEBGtest_basic (dropshot_external): successfully registered DTrace USDT probes local_addr = 127.0.0.1:49597
26182025-04-30T06:14:15.048ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = / versions = all
26192025-04-30T06:14:15.048ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /access versions = all
26202025-04-30T06:14:15.048ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /assets/{path} versions = all
26212025-04-30T06:14:15.048ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /device/auth versions = all
26222025-04-30T06:14:15.048ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /device/confirm versions = all
26232025-04-30T06:14:15.048ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /device/success versions = all
26242025-04-30T06:14:15.048ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /device/token versions = all
26252025-04-30T06:14:15.048ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /device/verify versions = all
26262025-04-30T06:14:15.049ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /experimental/v1/probes versions = all
26272025-04-30T06:14:15.049ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /experimental/v1/probes versions = all
26282025-04-30T06:14:15.049ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = DELETE path = /experimental/v1/probes/{probe} versions = all
26292025-04-30T06:14:15.049ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /experimental/v1/probes/{probe} versions = all
26302025-04-30T06:14:15.049ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /experimental/v1/system/support-bundles versions = all
26312025-04-30T06:14:15.050ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /experimental/v1/system/support-bundles versions = all
26322025-04-30T06:14:15.050ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = DELETE path = /experimental/v1/system/support-bundles/{support_bundle} versions = all
26332025-04-30T06:14:15.050ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /experimental/v1/system/support-bundles/{support_bundle} versions = all
26342025-04-30T06:14:15.050ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /experimental/v1/system/support-bundles/{support_bundle}/download versions = all
26352025-04-30T06:14:15.050ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = HEAD path = /experimental/v1/system/support-bundles/{support_bundle}/download versions = all
26362025-04-30T06:14:15.050ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /experimental/v1/system/support-bundles/{support_bundle}/download/{file} versions = all
26372025-04-30T06:14:15.050ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = HEAD path = /experimental/v1/system/support-bundles/{support_bundle}/download/{file} versions = all
26382025-04-30T06:14:15.051ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /experimental/v1/system/support-bundles/{support_bundle}/index versions = all
26392025-04-30T06:14:15.051ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /images versions = all
26402025-04-30T06:14:15.051ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /login versions = all
26412025-04-30T06:14:15.051ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /login/{silo_name}/local versions = all
26422025-04-30T06:14:15.051ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /login/{silo_name}/saml/{provider_name} versions = all
26432025-04-30T06:14:15.051ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /login/{silo_name}/saml/{provider_name} versions = all
26442025-04-30T06:14:15.051ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /login/{silo_name}/saml/{provider_name}/redirect versions = all
26452025-04-30T06:14:15.051ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /lookup/{path} versions = all
26462025-04-30T06:14:15.051ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /projects/{path} versions = all
26472025-04-30T06:14:15.051ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /projects-new versions = all
26482025-04-30T06:14:15.051ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /settings/{path} versions = all
26492025-04-30T06:14:15.051ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /system/{path} versions = all
26502025-04-30T06:14:15.051ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /utilization versions = all
26512025-04-30T06:14:15.051ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/affinity-groups versions = all
26522025-04-30T06:14:15.051ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/affinity-groups versions = all
26532025-04-30T06:14:15.051ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = DELETE path = /v1/affinity-groups/{affinity_group} versions = all
26542025-04-30T06:14:15.051ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/affinity-groups/{affinity_group} versions = all
26552025-04-30T06:14:15.051ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = PUT path = /v1/affinity-groups/{affinity_group} versions = all
26562025-04-30T06:14:15.051ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/affinity-groups/{affinity_group}/members versions = all
26572025-04-30T06:14:15.051ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = DELETE path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
26582025-04-30T06:14:15.051ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
26592025-04-30T06:14:15.051ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
26602025-04-30T06:14:15.051ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/anti-affinity-groups versions = all
26612025-04-30T06:14:15.051ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/anti-affinity-groups versions = all
26622025-04-30T06:14:15.051ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = DELETE path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
26632025-04-30T06:14:15.051ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
26642025-04-30T06:14:15.051ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = PUT path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
26652025-04-30T06:14:15.051ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group}/members versions = all
26662025-04-30T06:14:15.052ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = DELETE path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
26672025-04-30T06:14:15.052ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
26682025-04-30T06:14:15.052ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
26692025-04-30T06:14:15.052ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/certificates versions = all
26702025-04-30T06:14:15.052ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/certificates versions = all
26712025-04-30T06:14:15.052ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = DELETE path = /v1/certificates/{certificate} versions = all
26722025-04-30T06:14:15.052ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/certificates/{certificate} versions = all
26732025-04-30T06:14:15.052ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/disks versions = all
26742025-04-30T06:14:15.052ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/disks versions = all
26752025-04-30T06:14:15.052ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = DELETE path = /v1/disks/{disk} versions = all
26762025-04-30T06:14:15.052ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/disks/{disk} versions = all
26772025-04-30T06:14:15.052ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/disks/{disk}/bulk-write versions = all
26782025-04-30T06:14:15.052ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/disks/{disk}/bulk-write-start versions = all
26792025-04-30T06:14:15.052ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/disks/{disk}/bulk-write-stop versions = all
26802025-04-30T06:14:15.052ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/disks/{disk}/finalize versions = all
26812025-04-30T06:14:15.052ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/disks/{disk}/metrics/{metric} versions = all
26822025-04-30T06:14:15.052ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/floating-ips versions = all
26832025-04-30T06:14:15.052ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/floating-ips versions = all
26842025-04-30T06:14:15.052ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = DELETE path = /v1/floating-ips/{floating_ip} versions = all
26852025-04-30T06:14:15.052ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/floating-ips/{floating_ip} versions = all
26862025-04-30T06:14:15.052ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = PUT path = /v1/floating-ips/{floating_ip} versions = all
26872025-04-30T06:14:15.052ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/floating-ips/{floating_ip}/attach versions = all
26882025-04-30T06:14:15.052ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/floating-ips/{floating_ip}/detach versions = all
26892025-04-30T06:14:15.052ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/groups versions = all
26902025-04-30T06:14:15.052ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/groups/{group_id} versions = all
26912025-04-30T06:14:15.052ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/images versions = all
26922025-04-30T06:14:15.052ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/images versions = all
26932025-04-30T06:14:15.052ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = DELETE path = /v1/images/{image} versions = all
26942025-04-30T06:14:15.052ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/images/{image} versions = all
26952025-04-30T06:14:15.052ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/images/{image}/demote versions = all
26962025-04-30T06:14:15.052ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/images/{image}/promote versions = all
26972025-04-30T06:14:15.052ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/instances versions = all
26982025-04-30T06:14:15.052ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/instances versions = all
26992025-04-30T06:14:15.052ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = DELETE path = /v1/instances/{instance} versions = all
27002025-04-30T06:14:15.052ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/instances/{instance} versions = all
27012025-04-30T06:14:15.052ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = PUT path = /v1/instances/{instance} versions = all
27022025-04-30T06:14:15.052ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/instances/{instance}/affinity-groups versions = all
27032025-04-30T06:14:15.052ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/instances/{instance}/anti-affinity-groups versions = all
27042025-04-30T06:14:15.052ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/instances/{instance}/disks versions = all
27052025-04-30T06:14:15.053ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/instances/{instance}/disks/attach versions = all
27062025-04-30T06:14:15.053ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/instances/{instance}/disks/detach versions = all
27072025-04-30T06:14:15.053ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/instances/{instance}/external-ips versions = all
27082025-04-30T06:14:15.053ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = DELETE path = /v1/instances/{instance}/external-ips/ephemeral versions = all
27092025-04-30T06:14:15.053ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/instances/{instance}/external-ips/ephemeral versions = all
27102025-04-30T06:14:15.053ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/instances/{instance}/reboot versions = all
27112025-04-30T06:14:15.053ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/instances/{instance}/serial-console versions = all
27122025-04-30T06:14:15.053ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/instances/{instance}/serial-console/stream versions = all
27132025-04-30T06:14:15.053ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/instances/{instance}/ssh-public-keys versions = all
27142025-04-30T06:14:15.053ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/instances/{instance}/start versions = all
27152025-04-30T06:14:15.053ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/instances/{instance}/stop versions = all
27162025-04-30T06:14:15.053ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/internet-gateway-ip-addresses versions = all
27172025-04-30T06:14:15.053ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/internet-gateway-ip-addresses versions = all
27182025-04-30T06:14:15.053ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = DELETE path = /v1/internet-gateway-ip-addresses/{address} versions = all
27192025-04-30T06:14:15.053ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/internet-gateway-ip-pools versions = all
27202025-04-30T06:14:15.053ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/internet-gateway-ip-pools versions = all
27212025-04-30T06:14:15.053ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = DELETE path = /v1/internet-gateway-ip-pools/{pool} versions = all
27222025-04-30T06:14:15.053ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/internet-gateways versions = all
27232025-04-30T06:14:15.053ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/internet-gateways versions = all
27242025-04-30T06:14:15.053ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = DELETE path = /v1/internet-gateways/{gateway} versions = all
27252025-04-30T06:14:15.053ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/internet-gateways/{gateway} versions = all
27262025-04-30T06:14:15.053ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/ip-pools versions = all
27272025-04-30T06:14:15.053ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/ip-pools/{pool} versions = all
27282025-04-30T06:14:15.053ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/login/{silo_name}/local versions = all
27292025-04-30T06:14:15.053ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/logout versions = all
27302025-04-30T06:14:15.053ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/me versions = all
27312025-04-30T06:14:15.053ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/me/groups versions = all
27322025-04-30T06:14:15.053ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/me/ssh-keys versions = all
27332025-04-30T06:14:15.053ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/me/ssh-keys versions = all
27342025-04-30T06:14:15.053ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = DELETE path = /v1/me/ssh-keys/{ssh_key} versions = all
27352025-04-30T06:14:15.053ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/me/ssh-keys/{ssh_key} versions = all
27362025-04-30T06:14:15.053ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/metrics/{metric_name} versions = all
27372025-04-30T06:14:15.053ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/network-interfaces versions = all
27382025-04-30T06:14:15.053ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/network-interfaces versions = all
27392025-04-30T06:14:15.053ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = DELETE path = /v1/network-interfaces/{interface} versions = all
27402025-04-30T06:14:15.053ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/network-interfaces/{interface} versions = all
27412025-04-30T06:14:15.053ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = PUT path = /v1/network-interfaces/{interface} versions = all
27422025-04-30T06:14:15.053ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/ping versions = all
27432025-04-30T06:14:15.054ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/policy versions = all
27442025-04-30T06:14:15.054ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = PUT path = /v1/policy versions = all
27452025-04-30T06:14:15.054ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/projects versions = all
27462025-04-30T06:14:15.054ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/projects versions = all
27472025-04-30T06:14:15.054ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = DELETE path = /v1/projects/{project} versions = all
27482025-04-30T06:14:15.054ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/projects/{project} versions = all
27492025-04-30T06:14:15.054ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = PUT path = /v1/projects/{project} versions = all
27502025-04-30T06:14:15.054ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/projects/{project}/policy versions = all
27512025-04-30T06:14:15.054ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = PUT path = /v1/projects/{project}/policy versions = all
27522025-04-30T06:14:15.054ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/snapshots versions = all
27532025-04-30T06:14:15.054ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/snapshots versions = all
27542025-04-30T06:14:15.054ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = DELETE path = /v1/snapshots/{snapshot} versions = all
27552025-04-30T06:14:15.054ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/snapshots/{snapshot} versions = all
27562025-04-30T06:14:15.054ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/hardware/disks versions = all
27572025-04-30T06:14:15.054ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/hardware/disks/{disk_id} versions = all
27582025-04-30T06:14:15.054ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/hardware/rack-switch-port/{rack_id}/{switch_location}/{port}/lldp/neighbors versions = all
27592025-04-30T06:14:15.054ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/hardware/racks versions = all
27602025-04-30T06:14:15.054ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/hardware/racks/{rack_id} versions = all
27612025-04-30T06:14:15.054ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/hardware/sleds versions = all
27622025-04-30T06:14:15.054ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/system/hardware/sleds versions = all
27632025-04-30T06:14:15.054ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/hardware/sleds/{sled_id} versions = all
27642025-04-30T06:14:15.054ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/hardware/sleds/{sled_id}/disks versions = all
27652025-04-30T06:14:15.054ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/hardware/sleds/{sled_id}/instances versions = all
27662025-04-30T06:14:15.054ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = PUT path = /v1/system/hardware/sleds/{sled_id}/provision-policy versions = all
27672025-04-30T06:14:15.054ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/hardware/sleds-uninitialized versions = all
27682025-04-30T06:14:15.054ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/hardware/switch-port versions = all
27692025-04-30T06:14:15.054ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/hardware/switch-port/{port}/lldp/config versions = all
27702025-04-30T06:14:15.054ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/system/hardware/switch-port/{port}/lldp/config versions = all
27712025-04-30T06:14:15.054ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = DELETE path = /v1/system/hardware/switch-port/{port}/settings versions = all
27722025-04-30T06:14:15.054ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/system/hardware/switch-port/{port}/settings versions = all
27732025-04-30T06:14:15.054ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/hardware/switch-port/{port}/status versions = all
27742025-04-30T06:14:15.054ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/hardware/switches versions = all
27752025-04-30T06:14:15.054ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/hardware/switches/{switch_id} versions = all
27762025-04-30T06:14:15.054ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/identity-providers versions = all
27772025-04-30T06:14:15.054ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/system/identity-providers/local/users versions = all
27782025-04-30T06:14:15.054ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = DELETE path = /v1/system/identity-providers/local/users/{user_id} versions = all
27792025-04-30T06:14:15.054ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/system/identity-providers/local/users/{user_id}/set-password versions = all
27802025-04-30T06:14:15.054ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/system/identity-providers/saml versions = all
27812025-04-30T06:14:15.054ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/identity-providers/saml/{provider} versions = all
27822025-04-30T06:14:15.055ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/ip-pools versions = all
27832025-04-30T06:14:15.055ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/system/ip-pools versions = all
27842025-04-30T06:14:15.055ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = DELETE path = /v1/system/ip-pools/{pool} versions = all
27852025-04-30T06:14:15.055ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/ip-pools/{pool} versions = all
27862025-04-30T06:14:15.055ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = PUT path = /v1/system/ip-pools/{pool} versions = all
27872025-04-30T06:14:15.055ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/ip-pools/{pool}/ranges versions = all
27882025-04-30T06:14:15.055ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/system/ip-pools/{pool}/ranges/add versions = all
27892025-04-30T06:14:15.055ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/system/ip-pools/{pool}/ranges/remove versions = all
27902025-04-30T06:14:15.055ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/ip-pools/{pool}/silos versions = all
27912025-04-30T06:14:15.055ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/system/ip-pools/{pool}/silos versions = all
27922025-04-30T06:14:15.055ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = DELETE path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
27932025-04-30T06:14:15.055ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = PUT path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
27942025-04-30T06:14:15.055ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/ip-pools/{pool}/utilization versions = all
27952025-04-30T06:14:15.055ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/ip-pools-service versions = all
27962025-04-30T06:14:15.055ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/ip-pools-service/ranges versions = all
27972025-04-30T06:14:15.055ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/system/ip-pools-service/ranges/add versions = all
27982025-04-30T06:14:15.055ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/system/ip-pools-service/ranges/remove versions = all
27992025-04-30T06:14:15.055ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/metrics/{metric_name} versions = all
28002025-04-30T06:14:15.055ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/networking/address-lot versions = all
28012025-04-30T06:14:15.055ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/system/networking/address-lot versions = all
28022025-04-30T06:14:15.055ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = DELETE path = /v1/system/networking/address-lot/{address_lot} versions = all
28032025-04-30T06:14:15.055ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/networking/address-lot/{address_lot}/blocks versions = all
28042025-04-30T06:14:15.055ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/networking/allow-list versions = all
28052025-04-30T06:14:15.055ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = PUT path = /v1/system/networking/allow-list versions = all
28062025-04-30T06:14:15.055ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/system/networking/bfd-disable versions = all
28072025-04-30T06:14:15.055ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/system/networking/bfd-enable versions = all
28082025-04-30T06:14:15.055ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/networking/bfd-status versions = all
28092025-04-30T06:14:15.055ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = DELETE path = /v1/system/networking/bgp versions = all
28102025-04-30T06:14:15.055ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/networking/bgp versions = all
28112025-04-30T06:14:15.055ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/system/networking/bgp versions = all
28122025-04-30T06:14:15.055ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/networking/bgp-announce-set versions = all
28132025-04-30T06:14:15.055ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = PUT path = /v1/system/networking/bgp-announce-set versions = all
28142025-04-30T06:14:15.055ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = DELETE path = /v1/system/networking/bgp-announce-set/{announce_set} versions = all
28152025-04-30T06:14:15.055ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/networking/bgp-announce-set/{announce_set}/announcement versions = all
28162025-04-30T06:14:15.055ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/networking/bgp-exported versions = all
28172025-04-30T06:14:15.055ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/networking/bgp-message-history versions = all
28182025-04-30T06:14:15.055ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/networking/bgp-routes-ipv4 versions = all
28192025-04-30T06:14:15.055ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/networking/bgp-status versions = all
28202025-04-30T06:14:15.055ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/networking/loopback-address versions = all
28212025-04-30T06:14:15.055ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/system/networking/loopback-address versions = all
28222025-04-30T06:14:15.055ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = DELETE path = /v1/system/networking/loopback-address/{rack_id}/{switch_location}/{address}/{subnet_mask} versions = all
28232025-04-30T06:14:15.056ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = DELETE path = /v1/system/networking/switch-port-settings versions = all
28242025-04-30T06:14:15.056ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/networking/switch-port-settings versions = all
28252025-04-30T06:14:15.056ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/system/networking/switch-port-settings versions = all
28262025-04-30T06:14:15.056ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/networking/switch-port-settings/{port} versions = all
28272025-04-30T06:14:15.056ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/policy versions = all
28282025-04-30T06:14:15.056ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = PUT path = /v1/system/policy versions = all
28292025-04-30T06:14:15.056ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/roles versions = all
28302025-04-30T06:14:15.056ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/roles/{role_name} versions = all
28312025-04-30T06:14:15.056ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/silo-quotas versions = all
28322025-04-30T06:14:15.056ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/silos versions = all
28332025-04-30T06:14:15.056ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/system/silos versions = all
28342025-04-30T06:14:15.056ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = DELETE path = /v1/system/silos/{silo} versions = all
28352025-04-30T06:14:15.056ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/silos/{silo} versions = all
28362025-04-30T06:14:15.056ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/silos/{silo}/ip-pools versions = all
28372025-04-30T06:14:15.056ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/silos/{silo}/policy versions = all
28382025-04-30T06:14:15.056ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = PUT path = /v1/system/silos/{silo}/policy versions = all
28392025-04-30T06:14:15.056ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/silos/{silo}/quotas versions = all
28402025-04-30T06:14:15.056ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = PUT path = /v1/system/silos/{silo}/quotas versions = all
28412025-04-30T06:14:15.056ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/system/timeseries/query versions = all
28422025-04-30T06:14:15.056ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/timeseries/schemas versions = all
28432025-04-30T06:14:15.056ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = PUT path = /v1/system/update/repository versions = all
28442025-04-30T06:14:15.056ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/update/repository/{system_version} versions = all
28452025-04-30T06:14:15.056ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/update/target-release versions = all
28462025-04-30T06:14:15.056ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = PUT path = /v1/system/update/target-release versions = all
28472025-04-30T06:14:15.056ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/users versions = all
28482025-04-30T06:14:15.056ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/users/{user_id} versions = all
28492025-04-30T06:14:15.056ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/users-builtin versions = all
28502025-04-30T06:14:15.056ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/users-builtin/{user} versions = all
28512025-04-30T06:14:15.056ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/utilization/silos versions = all
28522025-04-30T06:14:15.056ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/system/utilization/silos/{silo} versions = all
28532025-04-30T06:14:15.057ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/timeseries/query versions = all
28542025-04-30T06:14:15.057ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/users versions = all
28552025-04-30T06:14:15.057ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/utilization versions = all
28562025-04-30T06:14:15.057ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/vpc-firewall-rules versions = all
28572025-04-30T06:14:15.057ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = PUT path = /v1/vpc-firewall-rules versions = all
28582025-04-30T06:14:15.057ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/vpc-router-routes versions = all
28592025-04-30T06:14:15.057ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/vpc-router-routes versions = all
28602025-04-30T06:14:15.057ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = DELETE path = /v1/vpc-router-routes/{route} versions = all
28612025-04-30T06:14:15.057ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/vpc-router-routes/{route} versions = all
28622025-04-30T06:14:15.057ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = PUT path = /v1/vpc-router-routes/{route} versions = all
28632025-04-30T06:14:15.057ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/vpc-routers versions = all
28642025-04-30T06:14:15.057ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/vpc-routers versions = all
28652025-04-30T06:14:15.057ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = DELETE path = /v1/vpc-routers/{router} versions = all
28662025-04-30T06:14:15.057ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/vpc-routers/{router} versions = all
28672025-04-30T06:14:15.057ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = PUT path = /v1/vpc-routers/{router} versions = all
28682025-04-30T06:14:15.057ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/vpc-subnets versions = all
28692025-04-30T06:14:15.057ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/vpc-subnets versions = all
28702025-04-30T06:14:15.057ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = DELETE path = /v1/vpc-subnets/{subnet} versions = all
28712025-04-30T06:14:15.057ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/vpc-subnets/{subnet} versions = all
28722025-04-30T06:14:15.057ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = PUT path = /v1/vpc-subnets/{subnet} versions = all
28732025-04-30T06:14:15.057ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/vpc-subnets/{subnet}/network-interfaces versions = all
28742025-04-30T06:14:15.057ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/vpcs versions = all
28752025-04-30T06:14:15.057ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/vpcs versions = all
28762025-04-30T06:14:15.057ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = DELETE path = /v1/vpcs/{vpc} versions = all
28772025-04-30T06:14:15.057ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/vpcs/{vpc} versions = all
28782025-04-30T06:14:15.057ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = PUT path = /v1/vpcs/{vpc} versions = all
28792025-04-30T06:14:15.057ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/webhooks/deliveries versions = all
28802025-04-30T06:14:15.057ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/webhooks/deliveries/{event_id}/resend versions = all
28812025-04-30T06:14:15.057ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/webhooks/event-classes versions = all
28822025-04-30T06:14:15.057ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/webhooks/receivers versions = all
28832025-04-30T06:14:15.057ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/webhooks/receivers versions = all
28842025-04-30T06:14:15.057ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = DELETE path = /v1/webhooks/receivers/{receiver} versions = all
28852025-04-30T06:14:15.057ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/webhooks/receivers/{receiver} versions = all
28862025-04-30T06:14:15.057ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = PUT path = /v1/webhooks/receivers/{receiver} versions = all
28872025-04-30T06:14:15.057ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/webhooks/receivers/{receiver}/probe versions = all
28882025-04-30T06:14:15.057ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/webhooks/receivers/{receiver}/subscriptions versions = all
28892025-04-30T06:14:15.058ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = DELETE path = /v1/webhooks/receivers/{receiver}/subscriptions/{subscription} versions = all
28902025-04-30T06:14:15.058ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = GET path = /v1/webhooks/secrets versions = all
28912025-04-30T06:14:15.058ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = POST path = /v1/webhooks/secrets versions = all
28922025-04-30T06:14:15.058ZDEBGtest_basic (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:63584 method = DELETE path = /v1/webhooks/secrets/{secret_id} versions = all
28932025-04-30T06:14:15.058ZINFOtest_basic (dropshot_external_techport): listening local_addr = 127.0.0.1:63584
28942025-04-30T06:14:15.058ZDEBGtest_basic (dropshot_external_techport): successfully registered DTrace USDT probes local_addr = 127.0.0.1:63584
28952025-04-30T06:14:15.058ZDEBGtest_basic (nexus-producer-server): registered DTrace probes
28962025-04-30T06:14:15.058ZDEBGtest_basic (dropshot): registered endpoint local_addr = 127.0.0.1:46907 method = GET path = /{producer_id} versions = all
28972025-04-30T06:14:15.058ZINFOtest_basic (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:328 local_addr = 127.0.0.1:46907
28982025-04-30T06:14:15.058ZDEBGtest_basic (dropshot): successfully registered DTrace USDT probes local_addr = 127.0.0.1:46907
28992025-04-30T06:14:15.058ZDEBGtest_basic (nexus-producer-server): Requested any available port, Dropshot server has been bound to 127.0.0.1:46907
29002025-04-30T06:14:15.058ZDEBGtest_basic (nexus-producer-server): Nexus IP provided explicitly, will use it addr = 127.0.0.1:34211
29012025-04-30T06:14:15.058ZDEBGtest_basic (nexus-producer-server): starting producer registration task
29022025-04-30T06:14:15.058ZINFOtest_basic (nexus-producer-server): starting oximeter metric producer server address = 127.0.0.1:46907 file = oximeter/producer/src/lib.rs:279 interval = 10s producer_id = 088ac96c-981b-4054-bea6-d8ab23ae1366
29032025-04-30T06:14:15.058ZDEBGtest_basic (init_with_steps): Running step start_oximeter
29042025-04-30T06:14:15.058ZDEBGtest_basic: Starting Oximeter
29052025-04-30T06:14:15.058ZDEBGtest_basic (oximeter): registered DTrace probes
29062025-04-30T06:14:15.058ZINFOtest_basic (oximeter): starting oximeter server file = oximeter/collector/src/lib.rs:236
29072025-04-30T06:14:15.058ZDEBGtest_basic (oximeter): creating ClickHouse client
29082025-04-30T06:14:15.059ZDEBGtest_basic (clickhouse-client): registered USDT probes collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812
29092025-04-30T06:14:15.060ZDEBGtest_basic (producer-registration-task): registering as oximeter producer with Nexus registration_count = 0
29102025-04-30T06:14:15.060ZDEBGtest_basic (producer-registration-task): will register with Nexus at 127.0.0.1:34211
29112025-04-30T06:14:15.143ZDEBGtest_basic (producer-registration-task): client request body = Some(Body) method = POST uri = http://127.0.0.1:34211/metrics/producers
29122025-04-30T06:14:15.145ZINFOtest_basic (dropshot_internal): accepted connection local_addr = 127.0.0.1:34211 remote_addr = 127.0.0.1:47199
29132025-04-30T06:14:15.152ZTRACtest_basic (dropshot_internal): incoming request local_addr = 127.0.0.1:34211 method = POST remote_addr = 127.0.0.1:47199 req_id = 7b1cfdeb-2e06-4e81-bf3c-136d8991eb8e uri = /metrics/producers
29142025-04-30T06:14:15.154ZTRACtest_basic (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = POST remote_addr = 127.0.0.1:47199 req_id = 7b1cfdeb-2e06-4e81-bf3c-136d8991eb8e resource = Database uri = /metrics/producers
29152025-04-30T06:14:15.154ZDEBGtest_basic (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = POST remote_addr = 127.0.0.1:47199 req_id = 7b1cfdeb-2e06-4e81-bf3c-136d8991eb8e roles = RoleSet { roles: {} } uri = /metrics/producers
29162025-04-30T06:14:15.155ZDEBGtest_basic (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = POST remote_addr = 127.0.0.1:47199 req_id = 7b1cfdeb-2e06-4e81-bf3c-136d8991eb8e resource = Database result = Ok(()) uri = /metrics/producers
29172025-04-30T06:14:15.167ZTRACtest_basic (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 sql = SELECT MAX(value) AS max_version FROM oximeter.version;
29182025-04-30T06:14:15.182ZWARNtest_basic (clickhouse-client): oximeter database does not exist, or is out-of-date collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/db/src/client/mod.rs:894 id = 852e9b0a-c261-4054-802e-b1d874643812
29192025-04-30T06:14:15.185ZDEBGtest_basic (oximeter-agent): oximeter database does not exist, creating collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
29202025-04-30T06:14:15.185ZINFOtest_basic (clickhouse-client): reading db version collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/db/src/client/mod.rs:794 id = 852e9b0a-c261-4054-802e-b1d874643812
29212025-04-30T06:14:15.186ZTRACtest_basic (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 sql = SELECT MAX(value) AS max_version FROM oximeter.version;
29222025-04-30T06:14:15.191ZWARNtest_basic (clickhouse-client): oximeter database does not exist, or is out-of-date collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/db/src/client/mod.rs:894 id = 852e9b0a-c261-4054-802e-b1d874643812
29232025-04-30T06:14:15.191ZINFOtest_basic (clickhouse-client): read oximeter database version collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/db/src/client/mod.rs:799 id = 852e9b0a-c261-4054-802e-b1d874643812 version = 0
29242025-04-30T06:14:15.191ZINFOtest_basic (clickhouse-client): initializing oximeter schema collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/db/src/client/mod.rs:804 id = 852e9b0a-c261-4054-802e-b1d874643812
29252025-04-30T06:14:15.191ZDEBGtest_basic (clickhouse-client): initializing ClickHouse database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812
29262025-04-30T06:14:15.192ZTRACtest_basic (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 sql = CREATE DATABASE IF NOT EXISTS oximeter
29272025-04-30T06:14:15.195ZTRACtest_basic (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 sql = \n\n/* The version table contains metadata about the `oximeter` database */\nCREATE TABLE IF NOT EXISTS oximeter.version\n(\n value UInt64,\n timestamp DateTime64(9, 'UTC')\n)\nENGINE = MergeTree()\nORDER BY (value, timestamp)
29282025-04-30T06:14:15.200ZTRACtest_basic (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 sql = \n\n/* The measurement tables contain all individual samples from each timeseries.\n *\n * Each table stores a single datum type, and otherwise contains nearly the same\n * structure. The primary sorting key is on the timeseries name, key, and then\n * timestamp, so that all timeseries from the same schema are grouped, followed\n * by all samples from the same timeseries.\n *\n * This reflects that one usually looks up the _key_ in one or more field table,\n * and then uses that to index quickly into the measurements tables.\n */\nCREATE TABLE IF NOT EXISTS oximeter.measurements_bool\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Bool)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29292025-04-30T06:14:15.204ZTRACtest_basic (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_i8\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Int8)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29302025-04-30T06:14:15.214ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b922598e-b012-4308-a2db-d84f8e721ded", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:15 GMT"} })
29312025-04-30T06:14:15.214ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=0
29322025-04-30T06:14:15.216ZINFOtest_basic (dropshot_internal): request completed error_message_external = Service Unavailable error_message_internal = no Oximeter instances available for assignment latency_us = 64069 local_addr = 127.0.0.1:34211 method = POST remote_addr = 127.0.0.1:47199 req_id = 7b1cfdeb-2e06-4e81-bf3c-136d8991eb8e response_code = 503 uri = /metrics/producers
29332025-04-30T06:14:15.217ZTRACtest_basic (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_u8\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(UInt8)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29342025-04-30T06:14:15.220ZDEBGtest_basic (producer-registration-task): client response result = Ok(Response { url: "http://127.0.0.1:34211/metrics/producers", status: 503, headers: {"content-type": "application/json", "x-request-id": "7b1cfdeb-2e06-4e81-bf3c-136d8991eb8e", "content-length": "133", "date": "Wed, 30 Apr 2025 06:14:15 GMT"} })
29352025-04-30T06:14:15.225ZWARNtest_basic (producer-registration-task): failed to register with Nexus, will retry count = 0 delay = 164.598171ms error = "Error Response: status: 503 Service Unavailable; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"7b1cfdeb-2e06-4e81-bf3c-136d8991eb8e\\", \\"content-length\\": \\"133\\", \\"date\\": \\"Wed, 30 Apr 2025 06:14:15 GMT\\"}; value: Error { error_code: Some(\\"ServiceNotAvailable\\"), message: \\"Service Unavailable\\", request_id: \\"7b1cfdeb-2e06-4e81-bf3c-136d8991eb8e\\" }" file = oximeter/producer/src/lib.rs:371
29362025-04-30T06:14:15.227ZTRACtest_basic (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_i16\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Int16)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29372025-04-30T06:14:15.234ZTRACtest_basic (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_u16\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(UInt16)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29382025-04-30T06:14:15.246ZTRACtest_basic (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_i32\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Int32)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29392025-04-30T06:14:15.249ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5b8101a4-d350-414a-8b12-9583caa78120", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:15 GMT"} })
29402025-04-30T06:14:15.250ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=1
29412025-04-30T06:14:15.250ZTRACtest_basic (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_u32\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(UInt32)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29422025-04-30T06:14:15.254ZTRACtest_basic (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_i64\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Int64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29432025-04-30T06:14:15.256ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "26f429a2-4cf1-4c39-a905-49ebf93d0729", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:15 GMT"} })
29442025-04-30T06:14:15.256ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cmpa
29452025-04-30T06:14:15.257ZTRACtest_basic (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_u64\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(UInt64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29462025-04-30T06:14:15.259ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "58bd998d-94ef-462d-9673-9ca15c9c756d", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:15 GMT"} })
29472025-04-30T06:14:15.260ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
29482025-04-30T06:14:15.261ZTRACtest_basic (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_f32\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Float32)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29492025-04-30T06:14:15.264ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "58f20b76-25ca-4467-8721-b472aa9a45ed", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:15 GMT"} })
29502025-04-30T06:14:15.264ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
29512025-04-30T06:14:15.270ZTRACtest_basic (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_f64\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29522025-04-30T06:14:15.275ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "659bc66a-50b6-42b2-8c66-fcccd1e74f88", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:15 GMT"} })
29532025-04-30T06:14:15.276ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
29542025-04-30T06:14:15.278ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b8caedbe-477b-4edb-b07c-1b330bb89995", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:15 GMT"} })
29552025-04-30T06:14:15.278ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0
29562025-04-30T06:14:15.281ZTRACtest_basic (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_string\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(String)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29572025-04-30T06:14:15.282ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "923771e7-707a-425a-9c2b-5a964b9dd402", "content-length": "723", "date": "Wed, 30 Apr 2025 06:14:15 GMT"} })
29582025-04-30T06:14:15.284ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=0
29592025-04-30T06:14:15.287ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d84397c6-ef76-485f-b98b-52608e885dc1", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:15 GMT"} })
29602025-04-30T06:14:15.288ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=1
29612025-04-30T06:14:15.288ZTRACtest_basic (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_bytes\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n /*\n * NOTE: Right now we can't unambiguously record a nullable byte array.\n * Arrays cannot be nested in `Nullable()` types, and encoding the array as\n * a string isn't palatable for a few reasons.\n * See: https://github.com/oxidecomputer/omicron/issues/4551 for more\n * details.\n */\n datum Array(UInt8)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29622025-04-30T06:14:15.298ZTRACtest_basic (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativei64\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Int64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29632025-04-30T06:14:15.298ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f78d16ff-5f9b-4a5f-9ad8-5a77032a8cbb", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:15 GMT"} })
29642025-04-30T06:14:15.298ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=0
29652025-04-30T06:14:15.310ZTRACtest_basic (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativeu64\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(UInt64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29662025-04-30T06:14:15.315ZTRACtest_basic (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativef32\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Float32)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29672025-04-30T06:14:15.319ZTRACtest_basic (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 sql = \n\n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativef64\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29682025-04-30T06:14:15.324ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5d617cbc-37b9-40bf-bba1-bfe31da6115b", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:15 GMT"} })
29692025-04-30T06:14:15.325ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=1
29702025-04-30T06:14:15.328ZTRACtest_basic (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogrami8\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n /*\n * NOTE: Array types cannot be Nullable, see\n * https://clickhouse.com/docs/en/sql-reference/data-types/nullable\n * for more details.\n *\n * This means we need to use empty arrays to indicate a missing value. This\n * is unfortunate, and at this point relies on the fact that an\n * `oximeter::Histogram` cannot have zero bins. If that changes, we'll need\n * to figure out another way to represent missing samples here.\n */\n bins Array(Int8),\n counts Array(UInt64),\n min Int8,\n max Int8,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29712025-04-30T06:14:15.332ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ae77e497-09be-4940-8a8c-3954a2050c07", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:15 GMT"} })
29722025-04-30T06:14:15.332ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=0
29732025-04-30T06:14:15.336ZTRACtest_basic (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogramu8\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(UInt8),\n counts Array(UInt64),\n min UInt8,\n max UInt8,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29742025-04-30T06:14:15.338ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9f315595-2f4d-46fc-9388-66563f6f3158", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:15 GMT"} })
29752025-04-30T06:14:15.338ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=1
29762025-04-30T06:14:15.354ZTRACtest_basic (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogrami16\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(Int16),\n counts Array(UInt64),\n min Int16,\n max Int16,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29772025-04-30T06:14:15.355ZDEBGtest_basic (producer-registration-task): will register with Nexus at 127.0.0.1:34211
29782025-04-30T06:14:15.500ZDEBGtest_basic (producer-registration-task): client request body = Some(Body) method = POST uri = http://127.0.0.1:34211/metrics/producers
29792025-04-30T06:14:15.501ZTRACtest_basic (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogramu16\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(UInt16),\n counts Array(UInt64),\n min UInt16,\n max UInt16,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29802025-04-30T06:14:15.507ZINFOtest_basic (dropshot_internal): accepted connection local_addr = 127.0.0.1:34211 remote_addr = 127.0.0.1:43477
29812025-04-30T06:14:15.510ZTRACtest_basic (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogrami32\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(Int32),\n counts Array(UInt64),\n min Int32,\n max Int32,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29822025-04-30T06:14:15.522ZTRACtest_basic (dropshot_internal): incoming request local_addr = 127.0.0.1:34211 method = POST remote_addr = 127.0.0.1:43477 req_id = 769cd608-3f51-462c-8c19-e739dd1bcd3f uri = /metrics/producers
29832025-04-30T06:14:15.524ZTRACtest_basic (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = POST remote_addr = 127.0.0.1:43477 req_id = 769cd608-3f51-462c-8c19-e739dd1bcd3f resource = Database uri = /metrics/producers
29842025-04-30T06:14:15.524ZDEBGtest_basic (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = POST remote_addr = 127.0.0.1:43477 req_id = 769cd608-3f51-462c-8c19-e739dd1bcd3f roles = RoleSet { roles: {} } uri = /metrics/producers
29852025-04-30T06:14:15.527ZDEBGtest_basic (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = POST remote_addr = 127.0.0.1:43477 req_id = 769cd608-3f51-462c-8c19-e739dd1bcd3f resource = Database result = Ok(()) uri = /metrics/producers
29862025-04-30T06:14:15.528ZTRACtest_basic (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogramu32\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(UInt32),\n counts Array(UInt64),\n min UInt32,\n max UInt32,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29872025-04-30T06:14:15.535ZTRACtest_basic (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogrami64\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(Int64),\n counts Array(UInt64),\n min Int64,\n max Int64,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29882025-04-30T06:14:15.542ZINFOtest_basic (dropshot_internal): request completed error_message_external = Service Unavailable error_message_internal = no Oximeter instances available for assignment latency_us = 20406 local_addr = 127.0.0.1:34211 method = POST remote_addr = 127.0.0.1:43477 req_id = 769cd608-3f51-462c-8c19-e739dd1bcd3f response_code = 503 uri = /metrics/producers
29892025-04-30T06:14:15.543ZTRACtest_basic (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogramu64\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(UInt64),\n counts Array(UInt64),\n min UInt64,\n max UInt64,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29902025-04-30T06:14:15.547ZDEBGtest_basic (producer-registration-task): client response result = Ok(Response { url: "http://127.0.0.1:34211/metrics/producers", status: 503, headers: {"content-type": "application/json", "x-request-id": "769cd608-3f51-462c-8c19-e739dd1bcd3f", "content-length": "133", "date": "Wed, 30 Apr 2025 06:14:15 GMT"} })
29912025-04-30T06:14:15.552ZWARNtest_basic (producer-registration-task): failed to register with Nexus, will retry count = 1 delay = 491.6697ms error = "Error Response: status: 503 Service Unavailable; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"769cd608-3f51-462c-8c19-e739dd1bcd3f\\", \\"content-length\\": \\"133\\", \\"date\\": \\"Wed, 30 Apr 2025 06:14:15 GMT\\"}; value: Error { error_code: Some(\\"ServiceNotAvailable\\"), message: \\"Service Unavailable\\", request_id: \\"769cd608-3f51-462c-8c19-e739dd1bcd3f\\" }" file = oximeter/producer/src/lib.rs:371
29922025-04-30T06:14:15.554ZTRACtest_basic (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogramf32\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(Float32),\n counts Array(UInt64),\n min Float32,\n max Float32,\n sum_of_samples Float64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29932025-04-30T06:14:15.556ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "04668112-cc1c-410e-9210-61cbeca7d198", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:15 GMT"} })
29942025-04-30T06:14:15.556ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cmpa
29952025-04-30T06:14:15.561ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "706565db-1bc5-4d0a-a633-d74106041d63", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:15 GMT"} })
29962025-04-30T06:14:15.561ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
29972025-04-30T06:14:15.564ZTRACtest_basic (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogramf64\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(Float64),\n counts Array(UInt64),\n min Float64,\n max Float64,\n sum_of_samples Float64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29982025-04-30T06:14:15.564ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9c7383f5-b3ae-48df-b210-e433a7f88088", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:15 GMT"} })
29992025-04-30T06:14:15.565ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
30002025-04-30T06:14:15.569ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "234234dc-edb3-4435-a576-bf11d3d2d17d", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:15 GMT"} })
30012025-04-30T06:14:15.569ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
30022025-04-30T06:14:15.569ZTRACtest_basic (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 sql = \n\n/* The field tables store named dimensions of each timeseries.\n *\n * As with the measurement tables, there is one field table for each field data\n * type. Fields are deduplicated by using the "replacing merge tree", though\n * this behavior **must not** be relied upon for query correctness.\n *\n * The index for the fields differs from the measurements, however. Rows are\n * sorted by timeseries name, then field name, field value, and finally\n * timeseries key. This reflects the most common pattern for looking them up:\n * by field name and possibly value, within a timeseries. The resulting keys are\n * usually then used to look up measurements.\n *\n * NOTE: We may want to consider a secondary index on these tables, sorting by\n * timeseries name and then key, since it would improve lookups where one\n * already has the key. Realistically though, these tables are quite small and\n * so performance benefits will be low in absolute terms.\n *\n * TTL: We use a materialized column to expire old field table records. This\n * column is generated automatically by the database whenever a new row is\n * inserted. It cannot be inserted directly, nor is it returned in a `SELECT *`\n * query. Since these tables are `ReplacingMergeTree`s, that means the last\n * record will remain during a deduplication, which will have the last\n * timestamp. ClickHouse will then expire old data for us, similar to the\n * measurement tables.\n */\nCREATE TABLE IF NOT EXISTS oximeter.fields_bool\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value Bool,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30032025-04-30T06:14:15.572ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cbdeb90b-c8fa-468f-b621-05d87c847489", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:15 GMT"} })
30042025-04-30T06:14:15.572ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1
30052025-04-30T06:14:15.575ZTRACtest_basic (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_i8\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value Int8,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30062025-04-30T06:14:15.576ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e37a4a32-8a81-4fee-8e1c-31f166011623", "content-length": "723", "date": "Wed, 30 Apr 2025 06:14:15 GMT"} })
30072025-04-30T06:14:15.576ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=0
30082025-04-30T06:14:15.580ZTRACtest_basic (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_u8\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value UInt8,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30092025-04-30T06:14:15.584ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ba4813f5-0de7-40ee-be95-5db3e7803d8f", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:15 GMT"} })
30102025-04-30T06:14:15.585ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=1
30112025-04-30T06:14:15.588ZTRACtest_basic (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_i16\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value Int16,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30122025-04-30T06:14:15.591ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a78aee3c-aef1-42ed-856b-abd518448ddf", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:15 GMT"} })
30132025-04-30T06:14:15.591ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=0
30142025-04-30T06:14:15.598ZTRACtest_basic (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_u16\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value UInt16,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30152025-04-30T06:14:15.603ZTRACtest_basic (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_i32\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value Int32,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30162025-04-30T06:14:15.603ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f7695c2d-0929-4b43-99db-21ac45e6edd3", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:15 GMT"} })
30172025-04-30T06:14:15.603ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=1
30182025-04-30T06:14:15.608ZTRACtest_basic (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_u32\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value UInt32,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30192025-04-30T06:14:15.609ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a328c5a-94a2-4dc1-8849-97d13ce0b23f", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:15 GMT"} })
30202025-04-30T06:14:15.609ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=0
30212025-04-30T06:14:15.613ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3dea4e6a-1e33-4d8a-a509-06e783802182", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:15 GMT"} })
30222025-04-30T06:14:15.613ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=1
30232025-04-30T06:14:15.614ZTRACtest_basic (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_i64\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value Int64,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30242025-04-30T06:14:15.622ZTRACtest_basic (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_u64\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value UInt64,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30252025-04-30T06:14:15.623ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "11473c5a-b5ca-4536-8d94-12aaa467da8b", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:15 GMT"} })
30262025-04-30T06:14:15.623ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cmpa
30272025-04-30T06:14:15.627ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d7f9e18c-7905-4b00-840f-8cd4a55aea73", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:15 GMT"} })
30282025-04-30T06:14:15.629ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
30292025-04-30T06:14:15.632ZTRACtest_basic (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_ipaddr\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value IPv6,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30302025-04-30T06:14:15.635ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "14d9fab6-0c3a-47e5-b7bc-4a1bc15eba83", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:15 GMT"} })
30312025-04-30T06:14:15.636ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
30322025-04-30T06:14:15.638ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5567867f-045e-48b2-80ea-bae0351dae2c", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:15 GMT"} })
30332025-04-30T06:14:15.638ZDEBGtest_basic (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
30342025-04-30T06:14:15.640ZTRACtest_basic (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_string\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value String,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30352025-04-30T06:14:15.642ZDEBGtest_basic (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7150415f-e089-4a61-8a83-961b78f25209", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:15 GMT"} })
30362025-04-30T06:14:15.642ZTRACtest_basic (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
30372025-04-30T06:14:15.642ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
30382025-04-30T06:14:15.642ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
30392025-04-30T06:14:15.642ZDEBGtest_basic (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
30402025-04-30T06:14:15.645ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
30412025-04-30T06:14:15.651ZTRACtest_basic (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_uuid\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value UUID,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30422025-04-30T06:14:15.653ZDEBGtest_basic (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30432025-04-30T06:14:15.673ZDEBGtest_basic (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
30442025-04-30T06:14:15.673ZTRACtest_basic (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
30452025-04-30T06:14:15.673ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
30462025-04-30T06:14:15.673ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
30472025-04-30T06:14:15.674ZDEBGtest_basic (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
30482025-04-30T06:14:15.675ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
30492025-04-30T06:14:15.677ZTRACtest_basic (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 sql = \n\n/* The timeseries schema table stores the extracted schema for the samples\n * oximeter collects.\n */\nCREATE TABLE IF NOT EXISTS oximeter.timeseries_schema\n(\n timeseries_name String,\n fields Nested(\n name String,\n type Enum(\n 'Bool' = 1,\n 'I64' = 2,\n 'IpAddr' = 3,\n 'String' = 4,\n 'Uuid' = 6,\n 'I8' = 7,\n 'U8' = 8,\n 'I16' = 9,\n 'U16' = 10,\n 'I32' = 11,\n 'U32' = 12,\n 'U64' = 13\n ),\n source Enum(\n 'Target' = 1,\n 'Metric' = 2\n )\n ),\n datum_type Enum(\n 'Bool' = 1,\n 'I64' = 2,\n 'F64' = 3,\n 'String' = 4,\n 'Bytes' = 5,\n 'CumulativeI64' = 6,\n 'CumulativeF64' = 7,\n 'HistogramI64' = 8,\n 'HistogramF64' = 9,\n 'I8' = 10,\n 'U8' = 11,\n 'I16' = 12,\n 'U16' = 13,\n 'I32' = 14,\n 'U32' = 15,\n 'U64' = 16,\n 'F32' = 17,\n 'CumulativeU64' = 18,\n 'CumulativeF32' = 19,\n 'HistogramI8' = 20,\n 'HistogramU8' = 21,\n 'HistogramI16' = 22,\n 'HistogramU16' = 23,\n 'HistogramI32' = 24,\n 'HistogramU32' = 25,\n 'HistogramU64' = 26,\n 'HistogramF32' = 27\n ),\n created DateTime64(9, 'UTC')\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, fields.name)
30502025-04-30T06:14:15.686ZINFOtest_basic (clickhouse-client): inserting current version collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/db/src/client/mod.rs:825 id = 852e9b0a-c261-4054-802e-b1d874643812 version = 14
30512025-04-30T06:14:15.686ZTRACtest_basic (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 sql = INSERT INTO oximeter.version (*) VALUES (14, now());
30522025-04-30T06:14:15.688ZDEBGtest_basic (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30532025-04-30T06:14:15.703ZDEBGtest_basic (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
30542025-04-30T06:14:15.703ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
30552025-04-30T06:14:15.703ZDEBGtest_basic (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
30562025-04-30T06:14:15.706ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
30572025-04-30T06:14:15.712ZDEBGtest_basic (clickhouse-client): registered USDT probes collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4e88c972-3e3c-4b5b-8f3d-e79743eb1cfe
30582025-04-30T06:14:15.714ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:41571 method = GET path = /info versions = all
30592025-04-30T06:14:15.714ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:41571 method = GET path = /producers versions = all
30602025-04-30T06:14:15.714ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:41571 method = DELETE path = /producers/{producer_id} versions = all
30612025-04-30T06:14:15.714ZDEBGtest_basic (dropshot): registered endpoint local_addr = [::1]:41571 method = GET path = /producers/{producer_id} versions = all
30622025-04-30T06:14:15.714ZINFOtest_basic (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:328 local_addr = [::1]:41571
30632025-04-30T06:14:15.716ZDEBGtest_basic (dropshot): successfully registered DTrace USDT probes local_addr = [::1]:41571
30642025-04-30T06:14:15.719ZDEBGtest_basic (oximeter): registered USDT probes
30652025-04-30T06:14:15.719ZDEBGtest_basic (oximeter): contacting nexus
30662025-04-30T06:14:16.624ZDEBGtest_basic (oximeter): client request body = Some(Body) method = POST uri = http://127.0.0.1:34211/metrics/collectors
30672025-04-30T06:14:16.626ZDEBGtest_basic (producer-registration-task): will register with Nexus at 127.0.0.1:34211
30682025-04-30T06:14:16.736ZDEBGtest_basic (producer-registration-task): client request body = Some(Body) method = POST uri = http://127.0.0.1:34211/metrics/producers
30692025-04-30T06:14:16.842ZDEBGtest_basic (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:42420
30702025-04-30T06:14:16.842ZDEBGtest_basic (BackgroundTasks): client request SledAgent = http://[::1]:42420 background_task = inventory_collection body = None method = GET uri = http://[::1]:42420/inventory
30712025-04-30T06:14:16.844ZINFOtest_basic (dropshot_internal): accepted connection local_addr = 127.0.0.1:34211 remote_addr = 127.0.0.1:35990
30722025-04-30T06:14:16.846ZINFOtest_basic (dropshot_internal): accepted connection local_addr = 127.0.0.1:34211 remote_addr = 127.0.0.1:37552
30732025-04-30T06:14:16.846ZINFOtest_basic (dropshot): accepted connection local_addr = [::1]:42420 remote_addr = [::1]:55284 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
30742025-04-30T06:14:16.846ZTRACtest_basic (results-sink): batch interval expired, but no samples to insert collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
30752025-04-30T06:14:16.846ZTRACtest_basic (results-sink-cluster): batch interval expired, but no samples to insert collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
30762025-04-30T06:14:16.848ZTRACtest_basic (dropshot_internal): incoming request local_addr = 127.0.0.1:34211 method = POST remote_addr = 127.0.0.1:37552 req_id = aef1a04a-41f8-4240-9952-4e9f34b9278c uri = /metrics/producers
30772025-04-30T06:14:16.848ZTRACtest_basic (dropshot): incoming request local_addr = [::1]:42420 method = GET remote_addr = [::1]:55284 req_id = 9d42ac81-7c1a-43f1-a9d7-f3d6ca798d33 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
30782025-04-30T06:14:16.849ZTRACtest_basic (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = POST remote_addr = 127.0.0.1:37552 req_id = aef1a04a-41f8-4240-9952-4e9f34b9278c resource = Database uri = /metrics/producers
30792025-04-30T06:14:16.849ZDEBGtest_basic (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = POST remote_addr = 127.0.0.1:37552 req_id = aef1a04a-41f8-4240-9952-4e9f34b9278c roles = RoleSet { roles: {} } uri = /metrics/producers
30802025-04-30T06:14:16.850ZDEBGtest_basic (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = POST remote_addr = 127.0.0.1:37552 req_id = aef1a04a-41f8-4240-9952-4e9f34b9278c resource = Database result = Ok(()) uri = /metrics/producers
30812025-04-30T06:14:16.851ZINFOtest_basic (dropshot): request completed latency_us = 2897 local_addr = [::1]:42420 method = GET remote_addr = [::1]:55284 req_id = 9d42ac81-7c1a-43f1-a9d7-f3d6ca798d33 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
30822025-04-30T06:14:16.853ZTRACtest_basic (dropshot_internal): incoming request local_addr = 127.0.0.1:34211 method = POST remote_addr = 127.0.0.1:35990 req_id = 8c8a9672-73d5-4b4f-a991-6ff1c4bb8fae uri = /metrics/collectors
30832025-04-30T06:14:16.853ZDEBGtest_basic (BackgroundTasks): client response SledAgent = http://[::1]:42420 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:42420/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "9d42ac81-7c1a-43f1-a9d7-f3d6ca798d33", "content-length": "2769", "date": "Wed, 30 Apr 2025 06:14:16 GMT"} })
30842025-04-30T06:14:16.855ZDEBGtest_basic (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
30852025-04-30T06:14:16.855ZDEBGtest_basic (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
30862025-04-30T06:14:16.855ZDEBGtest_basic (BackgroundTasks): finished collection background_task = inventory_collection
30872025-04-30T06:14:16.857ZTRACtest_basic (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
30882025-04-30T06:14:16.857ZTRACtest_basic (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
30892025-04-30T06:14:16.858ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
30902025-04-30T06:14:16.858ZDEBGtest_basic (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
30912025-04-30T06:14:16.871ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
30922025-04-30T06:14:16.871ZTRACtest_basic (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = POST remote_addr = 127.0.0.1:35990 req_id = 8c8a9672-73d5-4b4f-a991-6ff1c4bb8fae resource = Database uri = /metrics/collectors
30932025-04-30T06:14:16.871ZDEBGtest_basic (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = POST remote_addr = 127.0.0.1:35990 req_id = 8c8a9672-73d5-4b4f-a991-6ff1c4bb8fae roles = RoleSet { roles: {} } uri = /metrics/collectors
30942025-04-30T06:14:16.873ZDEBGtest_basic (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = POST remote_addr = 127.0.0.1:35990 req_id = 8c8a9672-73d5-4b4f-a991-6ff1c4bb8fae resource = Database result = Ok(()) uri = /metrics/collectors
30952025-04-30T06:14:16.907ZDEBGtest_basic (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30962025-04-30T06:14:16.913ZDEBGtest_basic (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
30972025-04-30T06:14:16.913ZTRACtest_basic (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
30982025-04-30T06:14:16.913ZDEBGtest_basic (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
30992025-04-30T06:14:16.914ZDEBGtest_basic (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
31002025-04-30T06:14:16.915ZINFOtest_basic (nexus): registered new oximeter metric collection server address = [::1]:41571 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78
31012025-04-30T06:14:16.915ZINFOtest_basic (dropshot_internal): request completed latency_us = 62725 local_addr = 127.0.0.1:34211 method = POST remote_addr = 127.0.0.1:35990 req_id = 8c8a9672-73d5-4b4f-a991-6ff1c4bb8fae response_code = 204 uri = /metrics/collectors
31022025-04-30T06:14:16.922ZINFOtest_basic (nexus): assigned collector to new producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 producer_id = 088ac96c-981b-4054-bea6-d8ab23ae1366
31032025-04-30T06:14:16.922ZINFOtest_basic (dropshot_internal): request completed latency_us = 74418 local_addr = 127.0.0.1:34211 method = POST remote_addr = 127.0.0.1:37552 req_id = aef1a04a-41f8-4240-9952-4e9f34b9278c response_code = 201 uri = /metrics/producers
31042025-04-30T06:14:16.924ZDEBGtest_basic (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:34211/metrics/collectors", status: 204, headers: {"x-request-id": "8c8a9672-73d5-4b4f-a991-6ff1c4bb8fae", "date": "Wed, 30 Apr 2025 06:14:16 GMT"} })
31052025-04-30T06:14:16.924ZINFOtest_basic (oximeter): oximeter registered with nexus file = oximeter/collector/src/lib.rs:392 id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78
31062025-04-30T06:14:16.924ZDEBGtest_basic (init_with_steps): Running step start_producer_server
31072025-04-30T06:14:16.924ZDEBGtest_basic: Starting test metric Producer Server
31082025-04-30T06:14:16.932ZINFOtest_basic: registering Oso class class = Action
31092025-04-30T06:14:16.932ZINFOtest_basic: registering Oso class class = AnyActor
31102025-04-30T06:14:16.933ZINFOtest_basic: registering Oso class class = AuthenticatedActor
31112025-04-30T06:14:16.933ZINFOtest_basic: registering Oso class class = BlueprintConfig
31122025-04-30T06:14:16.933ZINFOtest_basic: registering Oso class class = Database
31132025-04-30T06:14:16.933ZINFOtest_basic: registering Oso class class = DnsConfig
31142025-04-30T06:14:16.933ZINFOtest_basic: registering Oso class class = Fleet
31152025-04-30T06:14:16.933ZINFOtest_basic: registering Oso class class = Inventory
31162025-04-30T06:14:16.933ZINFOtest_basic: registering Oso class class = IpPoolList
31172025-04-30T06:14:16.933ZINFOtest_basic: registering Oso class class = ConsoleSessionList
31182025-04-30T06:14:16.933ZINFOtest_basic: registering Oso class class = DeviceAuthRequestList
31192025-04-30T06:14:16.933ZINFOtest_basic: registering Oso class class = SiloCertificateList
31202025-04-30T06:14:16.933ZINFOtest_basic: registering Oso class class = SiloIdentityProviderList
31212025-04-30T06:14:16.933ZINFOtest_basic: registering Oso class class = SiloUserList
31222025-04-30T06:14:16.933ZINFOtest_basic: registering Oso class class = TargetReleaseConfig
31232025-04-30T06:14:16.933ZINFOtest_basic: registering Oso class class = WebhookEventClassList
31242025-04-30T06:14:16.933ZINFOtest_basic: registering Oso class class = Project
31252025-04-30T06:14:16.933ZINFOtest_basic: registering Oso class class = Disk
31262025-04-30T06:14:16.933ZINFOtest_basic: registering Oso class class = Snapshot
31272025-04-30T06:14:16.933ZINFOtest_basic: registering Oso class class = ProjectImage
31282025-04-30T06:14:16.933ZINFOtest_basic: registering Oso class class = AffinityGroup
31292025-04-30T06:14:16.933ZINFOtest_basic: registering Oso class class = AntiAffinityGroup
31302025-04-30T06:14:16.933ZINFOtest_basic: registering Oso class class = Instance
31312025-04-30T06:14:16.933ZINFOtest_basic: registering Oso class class = IpPool
31322025-04-30T06:14:16.933ZINFOtest_basic: registering Oso class class = InstanceNetworkInterface
31332025-04-30T06:14:16.933ZINFOtest_basic: registering Oso class class = Vpc
31342025-04-30T06:14:16.933ZINFOtest_basic: registering Oso class class = VpcRouter
31352025-04-30T06:14:16.933ZINFOtest_basic: registering Oso class class = InternetGateway
31362025-04-30T06:14:16.933ZINFOtest_basic: registering Oso class class = InternetGatewayIpPool
31372025-04-30T06:14:16.933ZINFOtest_basic: registering Oso class class = InternetGatewayIpAddress
31382025-04-30T06:14:16.933ZINFOtest_basic: registering Oso class class = RouterRoute
31392025-04-30T06:14:16.933ZINFOtest_basic: registering Oso class class = VpcSubnet
31402025-04-30T06:14:16.933ZINFOtest_basic: registering Oso class class = FloatingIp
31412025-04-30T06:14:16.933ZINFOtest_basic: registering Oso class class = Image
31422025-04-30T06:14:16.933ZINFOtest_basic: registering Oso class class = SiloImage
31432025-04-30T06:14:16.933ZINFOtest_basic: registering Oso class class = AddressLot
31442025-04-30T06:14:16.933ZINFOtest_basic: registering Oso class class = Blueprint
31452025-04-30T06:14:16.933ZINFOtest_basic: registering Oso class class = LoopbackAddress
31462025-04-30T06:14:16.933ZINFOtest_basic: registering Oso class class = Certificate
31472025-04-30T06:14:16.933ZINFOtest_basic: registering Oso class class = ConsoleSession
31482025-04-30T06:14:16.933ZINFOtest_basic: registering Oso class class = DeviceAuthRequest
31492025-04-30T06:14:16.933ZINFOtest_basic: registering Oso class class = DeviceAccessToken
31502025-04-30T06:14:16.933ZINFOtest_basic: registering Oso class class = PhysicalDisk
31512025-04-30T06:14:16.933ZINFOtest_basic: registering Oso class class = Rack
31522025-04-30T06:14:16.934ZINFOtest_basic: registering Oso class class = RoleBuiltin
31532025-04-30T06:14:16.934ZINFOtest_basic: registering Oso class class = SshKey
31542025-04-30T06:14:16.934ZINFOtest_basic: registering Oso class class = Silo
31552025-04-30T06:14:16.934ZINFOtest_basic: registering Oso class class = SiloUser
31562025-04-30T06:14:16.934ZINFOtest_basic: registering Oso class class = SiloGroup
31572025-04-30T06:14:16.934ZINFOtest_basic: registering Oso class class = SupportBundle
31582025-04-30T06:14:16.934ZINFOtest_basic: registering Oso class class = IdentityProvider
31592025-04-30T06:14:16.934ZINFOtest_basic: registering Oso class class = SamlIdentityProvider
31602025-04-30T06:14:16.934ZINFOtest_basic: registering Oso class class = Sled
31612025-04-30T06:14:16.934ZINFOtest_basic: registering Oso class class = TufRepo
31622025-04-30T06:14:16.934ZINFOtest_basic: registering Oso class class = TufArtifact
31632025-04-30T06:14:16.934ZINFOtest_basic: registering Oso class class = WebhookEvent
31642025-04-30T06:14:16.934ZINFOtest_basic: registering Oso class class = WebhookReceiver
31652025-04-30T06:14:16.934ZINFOtest_basic: registering Oso class class = WebhookSecret
31662025-04-30T06:14:16.934ZINFOtest_basic: registering Oso class class = Zpool
31672025-04-30T06:14:16.934ZINFOtest_basic: registering Oso class class = Service
31682025-04-30T06:14:16.934ZINFOtest_basic: registering Oso class class = UserBuiltin
31692025-04-30T06:14:16.934ZINFOtest_basic: full Oso configuration config = #\n# Oso configuration for Omicron\n# This file is augmented by generated snippets.\n#\n\n#\n# ACTOR TYPES AND BASIC RULES\n#\n\n# `AnyActor` includes both authenticated and unauthenticated users.\nactor AnyActor {}\n\n# An `AuthenticatedActor` has an identity in the system. All of our operations\n# today require that an actor be authenticated.\nactor AuthenticatedActor {}\n\n# For any resource, `actor` can perform action `action` on it if they're\n# authenticated and their role(s) give them the corresponding permission on that\n# resource.\nallow(actor: AnyActor, action: Action, resource) if\n actor.authenticated and\n has_permission(actor.authn_actor.unwrap(), action.to_perm(), resource);\n\n# Define role relationships\nhas_role(actor: AuthenticatedActor, role: String, resource: Resource)\n\tif resource.has_role(actor, role);\n\n#\n# ROLES AND PERMISSIONS IN THE FLEET/SILO/PROJECT HIERARCHY\n#\n# We define the following permissions for most resources in the system:\n#\n# - "create_child": required to create child resources (of any type)\n#\n# - "list_children": required to list child resources (of all types) of a\n# resource\n#\n# - "modify": required to modify or delete a resource\n#\n# - "read": required to read a resource\n#\n# We define the following predefined roles for only a few high-level resources:\n# the Fleet (see below), Silo, Organization, and Project. The specific roles\n# are oriented around intended use-cases:\n#\n# - "admin": has all permissions on the resource\n#\n# - "collaborator": has "read", "list_children", and "create_child", plus\n# the "admin" role for child resources. The idea is that if you're an\n# Organization Collaborator, you have full control over the Projects within\n# the Organization, but you cannot modify or delete the Organization itself.\n#\n# - "viewer": has "read" and "list_children" on a resource\n#\n# Below the Project level, permissions are granted via roles at the Project\n# level. For example, for someone to be able to create, modify, or delete any\n# Instances, they must be granted project.collaborator, which means they can\n# create, modify, or delete _all_ resources in the Project.\n#\n# The complete set of predefined roles:\n#\n# - fleet.admin (superuser for the whole system)\n# - fleet.collaborator (can manage Silos)\n# - fleet.viewer (can read most non-siloed resources in the system)\n# - silo.admin (superuser for the silo)\n# - silo.collaborator (can create and own Organizations)\n# - silo.viewer (can read most resources within the Silo)\n# - organization.admin (complete control over an organization)\n# - organization.collaborator (can manage Projects)\n# - organization.viewer (can read most resources within the Organization)\n# - project.admin (complete control over a Project)\n# - project.collaborator (can manage all resources within the Project)\n# - project.viewer (can read most resources within the Project)\n#\n# Outside the Silo/Organization/Project hierarchy, we (currently) treat most\n# resources as nested under Fleet or else a synthetic resource (see below). We\n# do not yet support role assignments on anything other than Fleet, Silo,\n# Organization, or Project.\n#\n\n# "Fleet" is a global singleton representing the whole system. The name comes\n# from the idea described in RFD 24, but it's not quite right. This probably\n# should be more like "Region" or "AvailabilityZone". The precise boundaries\n# have not yet been figured out.\nresource Fleet {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\troles = [\n\t # Roles that can be attached by users\n\t "admin",\n\t "collaborator",\n\t "viewer",\n\n\t # Internal-only roles\n\t "external-authenticator"\n\t];\n\n\t# Roles implied by other roles on this resource\n\t"viewer" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\t"create_child" if "collaborator";\n\t"modify" if "admin";\n}\n\n# For fleets specifically, roles can be conferred by roles on the user's Silo.\nhas_role(actor: AuthenticatedActor, role: String, _: Fleet) if\n\tsilo_role in actor.confers_fleet_role(role) and\n\thas_role(actor, silo_role, actor.silo.unwrap());\n\nresource Silo {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\troles = [ "admin", "collaborator", "viewer" ];\n\n\t# Roles implied by other roles on this resource\n\t"viewer" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\n\t"create_child" if "collaborator";\n\t"modify" if "admin";\n\n\t# Permissions implied by roles on this resource's parent (Fleet). Fleet\n\t# privileges allow a user to see and potentially administer the Silo,\n\t# but they do not give anyone permission to look at anything inside the\n\t# Silo. To achieve this, we use permission rules here. (If we granted\n\t# Fleet administrators _roles_ on the Silo, then those would cascade\n\t# into the Silo as well.)\n\trelations = { parent_fleet: Fleet };\n\t"read" if "viewer" on "parent_fleet";\n\t"modify" if "collaborator" on "parent_fleet";\n\n\t# external authenticator has to create silo users\n\t"list_children" if "external-authenticator" on "parent_fleet";\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\n\nhas_relation(fleet: Fleet, "parent_fleet", silo: Silo)\n\tif silo.fleet = fleet;\n\n# As a special case, all authenticated users can read their own Silo. That's\n# not quite the same as having the "viewer" role. For example, they cannot list\n# Organizations in the Silo.\n#\n# One reason this is necessary is because if an unprivileged user tries to\n# create an Organization using "POST /organizations", they should get back a 403\n# (which implies they're able to see /organizations, which is essentially seeing\n# the Silo itself) rather than a 404. This behavior isn't a hard constraint\n# (i.e., you could reasonably get a 404 for an API you're not allowed to call).\n# Nor is the implementation (i.e., we could special-case this endpoint somehow).\n# But granting this permission is the simplest way to keep this endpoint's\n# behavior consistent with the rest of the API.\n#\n# This rule is also used to determine if a user can list the identity providers\n# in the Silo (which they should be able to), since that's predicated on being\n# able to read the Silo.\n#\n# It's unclear what else would break if users couldn't see their own Silo.\nhas_permission(actor: AuthenticatedActor, "read", silo: Silo)\n\tif silo in actor.silo;\n\nresource Project {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\troles = [ "admin", "collaborator", "viewer" ];\n\n\t# Roles implied by other roles on this resource\n\t"viewer" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\t"create_child" if "collaborator";\n\t"modify" if "admin";\n\n\t# Roles implied by roles on this resource's parent (Silo)\n\trelations = { parent_silo: Silo };\n\t"admin" if "collaborator" on "parent_silo";\n\t"viewer" if "viewer" on "parent_silo";\n}\nhas_relation(silo: Silo, "parent_silo", project: Project)\n\tif project.silo = silo;\n\n#\n# GENERAL RESOURCES OUTSIDE THE SILO/PROJECT HIERARCHY\n#\n# Many resources use snippets of Polar generated by the `authz_resource!` Rust\n# macro. Some resources require custom Polar code. Those appear here.\n#\n\nresource Certificate {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Fleet-level and silo-level roles both grant privileges on certificates.\n\t"read" if "admin" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"read" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", certificate: Certificate)\n\tif certificate.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", certificate: Certificate)\n\tif certificate.silo.fleet = fleet;\n\nresource SiloUser {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\t# Fleet and Silo administrators can manage a Silo's users. This is one\n\t# of the only areas of Silo configuration that Fleet Administrators have\n\t# permissions on.\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\t"list_children" if "read" on "parent_silo";\n\t"read" if "read" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\t"list_children" if "read" on "parent_fleet";\n\t"read" if "read" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", user: SiloUser)\n\tif user.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", user: SiloUser)\n\tif user.silo.fleet = fleet;\n\n# authenticated actors have all permissions on themselves\nhas_permission(actor: AuthenticatedActor, _perm: String, silo_user: SiloUser)\n if actor.equals_silo_user(silo_user);\n\nhas_permission(actor: AuthenticatedActor, "read", silo_user: SiloUser)\n if silo_user.silo in actor.silo;\n\nresource SiloGroup {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\trelations = { parent_silo: Silo };\n\t"list_children" if "read" on "parent_silo";\n\t"read" if "read" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n}\nhas_relation(silo: Silo, "parent_silo", group: SiloGroup)\n\tif group.silo = silo;\n\nresource SshKey {\n\tpermissions = [ "read", "modify" ];\n\trelations = { silo_user: SiloUser };\n\n\t"read" if "read" on "silo_user";\n\t"modify" if "modify" on "silo_user";\n}\nhas_relation(user: SiloUser, "silo_user", ssh_key: SshKey)\n\tif ssh_key.silo_user = user;\n\nresource IdentityProvider {\n\tpermissions = [\n\t "read",\n\t "modify",\n\t "create_child",\n\t "list_children",\n\t];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Silo-level roles grant privileges on identity providers.\n\t"read" if "viewer" on "parent_silo";\n\t"list_children" if "viewer" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\n\t# Fleet-level roles also grant privileges on identity providers.\n\t"read" if "viewer" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", identity_provider: IdentityProvider)\n\tif identity_provider.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: IdentityProvider)\n\tif collection.silo.fleet = fleet;\n\nresource SamlIdentityProvider {\n\tpermissions = [\n\t "read",\n\t "modify",\n\t "create_child",\n\t "list_children",\n\t];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Silo-level roles grant privileges on identity providers.\n\t"read" if "viewer" on "parent_silo";\n\t"list_children" if "viewer" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\n\t# Fleet-level roles also grant privileges on identity providers.\n\t"read" if "viewer" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", saml_identity_provider: SamlIdentityProvider)\n\tif saml_identity_provider.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SamlIdentityProvider)\n\tif collection.silo.fleet = fleet;\n\n#\n# SYNTHETIC RESOURCES OUTSIDE THE SILO HIERARCHY\n#\n# The resources here do not correspond to anything that appears explicitly in\n# the API or is stored in the database. These are used either at the top level\n# of the API path (e.g., "/v1/system/ip-pools") or as an implementation detail of the system\n# (in the case of console sessions and "Database"). The policies are\n# either statically-defined in this file or driven by role assignments on the\n# Fleet. None of these resources defines their own roles.\n#\n\n# Describes the policy for reading and modifying DNS configuration\n# (both internal and external)\nresource DnsConfig {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_fleet: Fleet };\n\t# "external-authenticator" requires these permissions because that's the\n\t# context that Nexus uses when creating and deleting Silos. These\n\t# operations necessarily need to read and modify DNS configuration.\n\t"read" if "external-authenticator" on "parent_fleet";\n\t"modify" if "external-authenticator" on "parent_fleet";\n\t# "admin" on the parent fleet also gets these permissions, primarily for\n\t# the test suite.\n\t"read" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", dns_config: DnsConfig)\n\tif dns_config.fleet = fleet;\n\n# Describes the policy for accessing blueprints\nresource BlueprintConfig {\n\tpermissions = [\n\t "list_children", # list blueprints\n\t "create_child", # create blueprint\n\t "read", # read the current target\n\t "modify", # change the current target\n\t];\n\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"read" if "viewer" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", list: BlueprintConfig)\n\tif list.fleet = fleet;\n\n# Describes the policy for accessing blueprints\nresource TargetReleaseConfig {\n\tpermissions = [\n\t "read", # read the current target release\n\t "modify", # change the current target release\n\t];\n\n\trelations = { parent_fleet: Fleet };\n\t"read" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", resource: TargetReleaseConfig)\n\tif resource.fleet = fleet;\n\n# Describes the policy for reading and modifying low-level inventory\nresource Inventory {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_fleet: Fleet };\n\t"read" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", inventory: Inventory)\n\tif inventory.fleet = fleet;\n\n# Describes the policy for accessing "/v1/system/ip-pools" in the API\nresource IpPoolList {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "create_child",\n\t];\n\n\t# Fleet Administrators can create or modify the IP Pools list.\n\trelations = { parent_fleet: Fleet };\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n\n\t# Fleet Viewers can list IP Pools\n\t"list_children" if "viewer" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", ip_pool_list: IpPoolList)\n\tif ip_pool_list.fleet = fleet;\n\n# Any authenticated user can create a child of a provided IP Pool.\n# This is necessary to use the pools when provisioning instances.\nhas_permission(actor: AuthenticatedActor, "create_child", ip_pool: IpPool)\n\tif silo in actor.silo and silo.fleet = ip_pool.fleet;\n\n# Describes the policy for creating and managing web console sessions.\nresource ConsoleSessionList {\n\tpermissions = [ "create_child" ];\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", collection: ConsoleSessionList)\n\tif collection.fleet = fleet;\n\n# Describes the policy for creating and managing device authorization requests.\nresource DeviceAuthRequestList {\n\tpermissions = [ "create_child" ];\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", collection: DeviceAuthRequestList)\n\tif collection.fleet = fleet;\n\n# Describes the policy for creating and managing Silo certificates\nresource SiloCertificateList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Both Fleet and Silo administrators can see and modify the Silo's\n\t# certificates.\n\t"list_children" if "admin" on "parent_silo";\n\t"list_children" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloCertificateList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloCertificateList)\n\tif collection.silo.fleet = fleet;\n\n# Describes the policy for creating and managing Silo identity providers\nresource SiloIdentityProviderList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Everyone who can read the Silo (which includes all the users in the\n\t# Silo) can see the identity providers in it.\n\t"list_children" if "read" on "parent_silo";\n\n\t# Fleet and Silo administrators can manage the Silo's identity provider\n\t# configuration. This is one of the only areas of Silo configuration\n\t# that Fleet Administrators have permissions on. This is also one of\n\t# the only cases where we need to look two levels up the hierarchy to\n\t# see if somebody has the right permission. For most other things,\n\t# permissions cascade down the hierarchy so we only need to look at the\n\t# parent.\n\t"create_child" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloIdentityProviderList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloIdentityProviderList)\n\tif collection.silo.fleet = fleet;\n\n# Describes the policy for creating and managing Silo users (mostly intended for\n# API-managed users)\nresource SiloUserList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Everyone who can read the Silo (which includes all the users in the\n\t# Silo) can see the users in it.\n\t"list_children" if "read" on "parent_silo";\n\n\t# Fleet and Silo administrators can manage the Silo's users. This is\n\t# one of the only areas of Silo configuration that Fleet Administrators\n\t# have permissions on. This is also one of the few cases (so far) where\n\t# we need to look two levels up the hierarchy to see if somebody has the\n\t# right permission. For most other things, permissions cascade down the\n\t# hierarchy so we only need to look at the parent.\n\t"create_child" if "admin" on "parent_silo";\n\t"list_children" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloUserList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloUserList)\n\tif collection.silo.fleet = fleet;\n\n# These rules grants the external authenticator role the permissions it needs to\n# read silo users and modify their sessions. This is necessary for login to\n# work.\nhas_permission(actor: AuthenticatedActor, "read", silo: Silo)\n\tif has_role(actor, "external-authenticator", silo.fleet);\nhas_permission(actor: AuthenticatedActor, "read", user: SiloUser)\n\tif has_role(actor, "external-authenticator", user.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", user: SiloUser)\n\tif has_role(actor, "external-authenticator", user.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "read", group: SiloGroup)\n\tif has_role(actor, "external-authenticator", group.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", group: SiloGroup)\n\tif has_role(actor, "external-authenticator", group.silo.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", session: ConsoleSession)\n\tif has_role(actor, "external-authenticator", session.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", session: ConsoleSession)\n\tif has_role(actor, "external-authenticator", session.fleet);\n\n# All authenticated users can read and delete device authn requests because\n# by necessity these operations happen before we've figured out what user (or\n# even Silo) the device auth is associated with. Any user can claim a device\n# auth request with the right user code (that's how it works) -- it's the user\n# code and associated logic that prevents unauthorized access here.\nhas_permission(_actor: AuthenticatedActor, "read", _device_auth: DeviceAuthRequest);\nhas_permission(_actor: AuthenticatedActor, "modify", _device_auth: DeviceAuthRequest);\n\nhas_permission(actor: AuthenticatedActor, "read", device_token: DeviceAccessToken)\n\tif has_role(actor, "external-authenticator", device_token.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", identity_provider: IdentityProvider)\n\tif has_role(actor, "external-authenticator", identity_provider.silo.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", saml_identity_provider: SamlIdentityProvider)\n\tif has_role(actor, "external-authenticator", saml_identity_provider.silo.fleet);\n\n# Describes the policy for who can access the internal database.\nresource Database {\n\tpermissions = [\n\t # "query" is required to perform any query against the database,\n\t # whether a read or write query. This is checked when an operation\n\t # checks out a database connection from the connection pool.\n\t #\n\t # Any authenticated user gets this permission. There's generally\n\t # some other authz check involved in the database query. For\n\t # example, if you're querying the database to "read" a "Project", we\n\t # should also be checking that. So why do we do this at all? It's\n\t # a belt-and-suspenders measure so that if we somehow introduced an\n\t # unauthenticated code path that hits the database, it cannot be\n\t # used to DoS the database because we won't allow the operation to\n\t # make the query. (As long as the code path _is_ authenticated, we\n\t # can use throttling mechanisms to prevent DoS.)\n\t "query",\n\n\t # "modify" is required to populate database data that's delivered\n\t # with the system. It should also be required for schema changes,\n\t # when we support those. This is separate from "query" so that we\n\t # cannot accidentally invoke these code paths from API calls and\n\t # other general functions.\n\t "modify"\n\t];\n}\n\n# All authenticated users have the "query" permission on the database.\nhas_permission(_actor: AuthenticatedActor, "query", _resource: Database);\n\n# The "db-init" user is the only one with the "modify" permission.\nhas_permission(USER_DB_INIT: AuthenticatedActor, "modify", _resource: Database);\nhas_permission(USER_DB_INIT: AuthenticatedActor, "create_child", _resource: IpPoolList);\n# It also has "admin" on the internal silo to populate it with built-in resources.\n# TODO-completeness: actually limit to just internal silo and not all silos\nhas_role(USER_DB_INIT: AuthenticatedActor, "admin", _silo: Silo);\n\n# Allow the internal API admin permissions on all silos.\nhas_role(USER_INTERNAL_API: AuthenticatedActor, "admin", _silo: Silo);\n\nresource WebhookSecret {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_webhook_receiver: WebhookReceiver };\n\n\t"read" if "read" on "parent_webhook_receiver";\n\t"modify" if "modify" on "parent_webhook_receiver";\n}\n\nhas_relation(rx: WebhookReceiver, "parent_webhook_receiver", secret: WebhookSecret)\n\tif secret.webhook_receiver = rx;\n\nresource WebhookEventClassList {\n\tpermissions = [ "list_children" ];\n\trelations = { parent_fleet: Fleet };\n\n\t"list_children" if "viewer" on "parent_fleet";\n}\n\nhas_relation(fleet: Fleet, "parent_fleet", collection: WebhookEventClassList)\n\tif collection.fleet = fleet;\n\n\n\n resource Disk {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Disk)\n if child.project = parent;\n \n\n resource Snapshot {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Snapshot)\n if child.project = parent;\n \n\n resource ProjectImage {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: ProjectImage)\n if child.project = parent;\n \n\n resource AffinityGroup {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: AffinityGroup)\n if child.project = parent;\n \n\n resource AntiAffinityGroup {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: AntiAffinityGroup)\n if child.project = parent;\n \n\n resource Instance {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Instance)\n if child.project = parent;\n \n\n resource IpPool {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: IpPool)\n if child.fleet = fleet;\n \n\n resource InstanceNetworkInterface {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Instance\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InstanceNetworkInterface)\n if has_relation(project, "containing_project", child.instance);\n\n has_relation(parent: Instance, "parent", child: InstanceNetworkInterface)\n if child.instance = parent;\n \n\n resource Vpc {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Vpc)\n if child.project = parent;\n \n\n resource VpcRouter {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Vpc\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: VpcRouter)\n if has_relation(project, "containing_project", child.vpc);\n\n has_relation(parent: Vpc, "parent", child: VpcRouter)\n if child.vpc = parent;\n \n\n resource InternetGateway {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Vpc\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InternetGateway)\n if has_relation(project, "containing_project", child.vpc);\n\n has_relation(parent: Vpc, "parent", child: InternetGateway)\n if child.vpc = parent;\n \n\n resource InternetGatewayIpPool {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: InternetGateway\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InternetGatewayIpPool)\n if has_relation(project, "containing_project", child.internet_gateway);\n\n has_relation(parent: InternetGateway, "parent", child: InternetGatewayIpPool)\n if child.internet_gateway = parent;\n \n\n resource InternetGatewayIpAddress {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: InternetGateway\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InternetGatewayIpAddress)\n if has_relation(project, "containing_project", child.internet_gateway);\n\n has_relation(parent: InternetGateway, "parent", child: InternetGatewayIpAddress)\n if child.internet_gateway = parent;\n \n\n resource RouterRoute {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: VpcRouter\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: RouterRoute)\n if has_relation(project, "containing_project", child.vpc_router);\n\n has_relation(parent: VpcRouter, "parent", child: RouterRoute)\n if child.vpc_router = parent;\n \n\n resource VpcSubnet {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Vpc\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: VpcSubnet)\n if has_relation(project, "containing_project", child.vpc);\n\n has_relation(parent: Vpc, "parent", child: VpcSubnet)\n if child.vpc = parent;\n \n\n resource FloatingIp {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: FloatingIp)\n if child.project = parent;\n \n\n resource Image {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_silo: Silo };\n "list_children" if "viewer" on "containing_silo";\n "read" if "viewer" on "containing_silo";\n "modify" if "collaborator" on "containing_silo";\n "create_child" if "collaborator" on "containing_silo";\n }\n\n has_relation(parent: Silo, "containing_silo", child: Image)\n if child.silo = parent;\n \n\n resource SiloImage {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_silo: Silo };\n "list_children" if "viewer" on "containing_silo";\n "read" if "viewer" on "containing_silo";\n "modify" if "collaborator" on "containing_silo";\n "create_child" if "collaborator" on "containing_silo";\n }\n\n has_relation(parent: Silo, "containing_silo", child: SiloImage)\n if child.silo = parent;\n \n\n resource AddressLot {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: AddressLot)\n if child.fleet = fleet;\n \n\n resource Blueprint {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Blueprint)\n if child.fleet = fleet;\n \n\n resource LoopbackAddress {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: LoopbackAddress)\n if child.fleet = fleet;\n \n\n\n resource ConsoleSession {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: ConsoleSession)\n if child.fleet = fleet;\n \n\n resource DeviceAuthRequest {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: DeviceAuthRequest)\n if child.fleet = fleet;\n \n\n resource DeviceAccessToken {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: DeviceAccessToken)\n if child.fleet = fleet;\n \n\n resource PhysicalDisk {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: PhysicalDisk)\n if child.fleet = fleet;\n \n\n resource Rack {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Rack)\n if child.fleet = fleet;\n \n\n resource RoleBuiltin {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: RoleBuiltin)\n if child.fleet = fleet;\n \n\n\n\n\n\n resource SupportBundle {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: SupportBundle)\n if child.fleet = fleet;\n \n\n\n\n resource Sled {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Sled)\n if child.fleet = fleet;\n \n\n resource TufRepo {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: TufRepo)\n if child.fleet = fleet;\n \n\n resource TufArtifact {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: TufArtifact)\n if child.fleet = fleet;\n \n\n resource WebhookEvent {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: WebhookEvent)\n if child.fleet = fleet;\n \n\n resource WebhookReceiver {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: WebhookReceiver)\n if child.fleet = fleet;\n \n\n\n resource Zpool {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Zpool)\n if child.fleet = fleet;\n \n\n resource Service {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Service)\n if child.fleet = fleet;\n \n\n resource UserBuiltin {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: UserBuiltin)\n if child.fleet = fleet;\n
31702025-04-30T06:14:16.971ZINFOtest_basic: new DNS resolver addresses = [[::1]:64728]
31712025-04-30T06:14:16.971ZTRACtest_basic: authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory
31722025-04-30T06:14:16.972ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
31732025-04-30T06:14:16.972ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
31742025-04-30T06:14:16.972ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
31752025-04-30T06:14:16.973ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
31762025-04-30T06:14:16.975ZINFOtest_basic (oximeter-agent): starting refresh list task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:559
31772025-04-30T06:14:17.092ZDEBGtest_basic (producer-registration-task): client response result = Ok(Response { url: "http://127.0.0.1:34211/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "aef1a04a-41f8-4240-9952-4e9f34b9278c", "content-length": "41", "date": "Wed, 30 Apr 2025 06:14:16 GMT"} })
31782025-04-30T06:14:17.095ZDEBGtest_basic (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s wait_period = 150s
31792025-04-30T06:14:17.098ZINFOtest_basic (dropshot_internal): accepted connection local_addr = 127.0.0.1:34211 remote_addr = 127.0.0.1:59366
31802025-04-30T06:14:17.099ZINFOtest_basic (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
31812025-04-30T06:14:17.105ZDEBGtest_basic (oximeter): client request body = None method = GET uri = http://127.0.0.1:34211/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
31822025-04-30T06:14:17.108ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31832025-04-30T06:14:17.120ZDEBGtest_basic: authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory result = Ok(())
31842025-04-30T06:14:17.120ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
31852025-04-30T06:14:17.120ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
31862025-04-30T06:14:17.122ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
31872025-04-30T06:14:17.125ZTRACtest_basic (dropshot_internal): incoming request local_addr = 127.0.0.1:34211 method = POST remote_addr = 127.0.0.1:59366 req_id = 87025596-7157-4047-87cb-456f31271e64 uri = /metrics/producers
31882025-04-30T06:14:17.126ZTRACtest_basic (dropshot_internal): incoming request local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 3f3cb1d8-146e-4ce6-b38f-48cbcf401482 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
31892025-04-30T06:14:17.126ZTRACtest_basic (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = POST remote_addr = 127.0.0.1:59366 req_id = 87025596-7157-4047-87cb-456f31271e64 resource = Database uri = /metrics/producers
31902025-04-30T06:14:17.127ZDEBGtest_basic (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = POST remote_addr = 127.0.0.1:59366 req_id = 87025596-7157-4047-87cb-456f31271e64 roles = RoleSet { roles: {} } uri = /metrics/producers
31912025-04-30T06:14:17.130ZDEBGtest_basic (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = POST remote_addr = 127.0.0.1:59366 req_id = 87025596-7157-4047-87cb-456f31271e64 resource = Database result = Ok(()) uri = /metrics/producers
31922025-04-30T06:14:17.131ZTRACtest_basic (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 3f3cb1d8-146e-4ce6-b38f-48cbcf401482 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
31932025-04-30T06:14:17.131ZDEBGtest_basic (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 3f3cb1d8-146e-4ce6-b38f-48cbcf401482 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
31942025-04-30T06:14:17.133ZDEBGtest_basic (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 3f3cb1d8-146e-4ce6-b38f-48cbcf401482 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
31952025-04-30T06:14:17.155ZINFOtest_basic (nexus): assigned collector to new producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
31962025-04-30T06:14:17.155ZINFOtest_basic (dropshot_internal): request completed latency_us = 30299 local_addr = 127.0.0.1:34211 method = POST remote_addr = 127.0.0.1:59366 req_id = 87025596-7157-4047-87cb-456f31271e64 response_code = 201 uri = /metrics/producers
31972025-04-30T06:14:17.163ZINFOtest_basic (dropshot_internal): request completed latency_us = 37446 local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 3f3cb1d8-146e-4ce6-b38f-48cbcf401482 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
31982025-04-30T06:14:17.177ZDEBGtest_basic (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:34211/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f3cb1d8-146e-4ce6-b38f-48cbcf401482", "content-length": "405", "date": "Wed, 30 Apr 2025 06:14:16 GMT"} })
31992025-04-30T06:14:17.177ZDEBGtest_basic (oximeter): client request body = None method = GET uri = http://127.0.0.1:34211/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
32002025-04-30T06:14:17.179ZTRACtest_basic (dropshot_internal): incoming request local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = f7c75fc3-5690-4a8d-ac35-0f2f1f1aadfd uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
32012025-04-30T06:14:17.180ZTRACtest_basic (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = f7c75fc3-5690-4a8d-ac35-0f2f1f1aadfd resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
32022025-04-30T06:14:17.180ZDEBGtest_basic (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = f7c75fc3-5690-4a8d-ac35-0f2f1f1aadfd roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
32032025-04-30T06:14:17.181ZDEBGtest_basic (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = f7c75fc3-5690-4a8d-ac35-0f2f1f1aadfd resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
32042025-04-30T06:14:17.210ZINFOtest_basic (dropshot_internal): request completed latency_us = 31363 local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = f7c75fc3-5690-4a8d-ac35-0f2f1f1aadfd response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
32052025-04-30T06:14:17.212ZDEBGtest_basic (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:34211/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "f7c75fc3-5690-4a8d-ac35-0f2f1f1aadfd", "content-length": "29", "date": "Wed, 30 Apr 2025 06:14:16 GMT"} })
32062025-04-30T06:14:17.213ZDEBGtest_basic (oximeter-agent): registered new metric producer address = 127.0.0.1:46907 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 088ac96c-981b-4054-bea6-d8ab23ae1366
32072025-04-30T06:14:17.349ZTRACtest_basic (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 088ac96c-981b-4054-bea6-d8ab23ae1366
32082025-04-30T06:14:17.352ZDEBGtest_basic (oximeter-agent): registered new metric producer address = [::1]:41018 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
32092025-04-30T06:14:17.523ZTRACtest_basic (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
32102025-04-30T06:14:17.533ZINFOtest_basic (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
32112025-04-30T06:14:17.569ZWARNtest_basic (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
32122025-04-30T06:14:17.571ZWARNtest_basic (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
32132025-04-30T06:14:17.571ZDEBGtest_basic (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 3.537725257s iteration = 3 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
32142025-04-30T06:14:17.572ZDEBGtest_basic: inventory_prune_one: nothing eligible for removal (too few) candidates = [(efcb0ee5-cafa-4692-9b75-fcee4732bdca, 0)]
32152025-04-30T06:14:17.572ZTRACtest_basic: lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
32162025-04-30T06:14:17.603ZTRACtest_basic (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: d9 e7 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:56796 req_id = ee0b6082-0256-4f48-80ed-70e3e438e6c5
32172025-04-30T06:14:17.603ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 55783,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:56796 req_id = ee0b6082-0256-4f48-80ed-70e3e438e6c5
32182025-04-30T06:14:17.604ZDEBGtest_basic (store): query key key = _mgs._tcp
32192025-04-30T06:14:17.604ZDEBGtest_basic (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
32202025-04-30T06:14:17.604ZDEBGtest_basic (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:56796 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 37822, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = ee0b6082-0256-4f48-80ed-70e3e438e6c5
32212025-04-30T06:14:17.605ZDEBGtest_basic: lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 37822, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2522, tv_nsec: 404099488 } })
32222025-04-30T06:14:17.605ZTRACtest_basic: lookup_service_targets: looking up SRV target
32232025-04-30T06:14:17.606ZTRACtest_basic (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 45 25 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e E%..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:56366 req_id = b8d35753-8e43-46ef-ad24-6f758b68ad9d
32242025-04-30T06:14:17.607ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 17701,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:56366 req_id = b8d35753-8e43-46ef-ad24-6f758b68ad9d
32252025-04-30T06:14:17.607ZDEBGtest_basic (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
32262025-04-30T06:14:17.607ZDEBGtest_basic (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:56366 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = b8d35753-8e43-46ef-ad24-6f758b68ad9d
32272025-04-30T06:14:17.693ZTRACtest_basic: lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
32282025-04-30T06:14:17.694ZTRACtest_basic (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 65 a7 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c e............_cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:58188 req_id = dfbffc53-fdda-4880-8f8e-6097db1d7dc5
32292025-04-30T06:14:17.694ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 26023,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:58188 req_id = dfbffc53-fdda-4880-8f8e-6097db1d7dc5
32302025-04-30T06:14:17.694ZDEBGtest_basic (store): query key key = _clickhouse-admin-keeper._tcp
32312025-04-30T06:14:17.694ZERROtest_basic (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 26023, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:58188 req_id = dfbffc53-fdda-4880-8f8e-6097db1d7dc5
32322025-04-30T06:14:17.694ZDEBGtest_basic: begin collection
32332025-04-30T06:14:17.694ZDEBGtest_basic: begin collection from MGS mgs_url = http://[::1]:37822
32342025-04-30T06:14:17.694ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/ignition
32352025-04-30T06:14:17.697ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "e73e8b98-e717-4ead-b8f0-e058120ccfab", "content-length": "835", "date": "Wed, 30 Apr 2025 06:14:16 GMT"} })
32362025-04-30T06:14:17.699ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0
32372025-04-30T06:14:17.702ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cdf11a75-355e-4ecf-9a74-7fe2f948896a", "content-length": "734", "date": "Wed, 30 Apr 2025 06:14:16 GMT"} })
32382025-04-30T06:14:17.702ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=0
32392025-04-30T06:14:17.712ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "67385e71-c615-4b2b-9b32-a081caba7c5f", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:16 GMT"} })
32402025-04-30T06:14:17.713ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=1
32412025-04-30T06:14:17.717ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5d93084a-06a4-4ef9-aa8c-fcaab99eadc1", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:16 GMT"} })
32422025-04-30T06:14:17.718ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=0
32432025-04-30T06:14:17.720ZTRACtest_basic (results-sink): batch interval expired, but no samples to insert collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
32442025-04-30T06:14:17.721ZTRACtest_basic (results-sink-cluster): batch interval expired, but no samples to insert collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
32452025-04-30T06:14:17.723ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4dc29a97-9974-4b1d-af0a-0afc4f74229f", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:16 GMT"} })
32462025-04-30T06:14:17.724ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=1
32472025-04-30T06:14:17.728ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8b571b13-5f2d-4c0e-b476-eb471d9dc8df", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:16 GMT"} })
32482025-04-30T06:14:17.728ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=0
32492025-04-30T06:14:17.737ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6ae61288-1ea6-496f-8c9c-9eab51efb4ea", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:16 GMT"} })
32502025-04-30T06:14:17.737ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=1
32512025-04-30T06:14:17.746ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1cf8fa24-77c7-42fe-9c35-e5825f4c8ec0", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:16 GMT"} })
32522025-04-30T06:14:17.747ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cmpa
32532025-04-30T06:14:17.751ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6247aae4-43e1-4a10-8226-efb480d1a311", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:16 GMT"} })
32542025-04-30T06:14:17.752ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
32552025-04-30T06:14:17.755ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3590af4b-7b47-4918-8cd0-5bd4794d0490", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:16 GMT"} })
32562025-04-30T06:14:17.755ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
32572025-04-30T06:14:17.759ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "27a41740-331e-4fa9-a1da-b36bcc76f49d", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:16 GMT"} })
32582025-04-30T06:14:17.761ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
32592025-04-30T06:14:17.764ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2a995ee9-0553-4b5a-9426-edb456e78a2d", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:16 GMT"} })
32602025-04-30T06:14:17.764ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1
32612025-04-30T06:14:17.767ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "15974a62-1aa1-4eea-b47e-a8257ff96463", "content-length": "734", "date": "Wed, 30 Apr 2025 06:14:16 GMT"} })
32622025-04-30T06:14:17.769ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=0
32632025-04-30T06:14:17.781ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "de7f0586-1bda-41b5-b1e1-520349af7af7", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:16 GMT"} })
32642025-04-30T06:14:17.781ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=1
32652025-04-30T06:14:17.787ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8c3baf96-7a90-46cd-9fa1-d6e1f69453b3", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:16 GMT"} })
32662025-04-30T06:14:17.787ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=0
32672025-04-30T06:14:17.789ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4b6ed4cf-9019-4956-9c06-c3a2550e4655", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:16 GMT"} })
32682025-04-30T06:14:17.790ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=1
32692025-04-30T06:14:17.792ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5853b546-9bae-483c-92e4-d48df6406827", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:16 GMT"} })
32702025-04-30T06:14:17.792ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=0
32712025-04-30T06:14:17.797ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ac9f58c1-089e-4c6e-a910-676803469b71", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:16 GMT"} })
32722025-04-30T06:14:17.797ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=1
32732025-04-30T06:14:17.811ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "55e2c177-7b72-4988-ab4e-adf841d9f388", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:16 GMT"} })
32742025-04-30T06:14:17.811ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cmpa
32752025-04-30T06:14:17.815ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "12a0397f-1740-42a1-b927-4934d80fce06", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:16 GMT"} })
32762025-04-30T06:14:17.816ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
32772025-04-30T06:14:17.819ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9ffbdb01-d9eb-41e3-b7e2-91b888014a8d", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:16 GMT"} })
32782025-04-30T06:14:17.819ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
32792025-04-30T06:14:17.827ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "94b22dfe-9e7e-4486-ab91-c41c7f28e8b1", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:16 GMT"} })
32802025-04-30T06:14:17.827ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
32812025-04-30T06:14:17.831ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fea09f58-aa20-497c-a512-ff3bf40692e1", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:16 GMT"} })
32822025-04-30T06:14:17.831ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0
32832025-04-30T06:14:17.833ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d21237f8-ccad-43f3-8f8d-42832e3a16c7", "content-length": "723", "date": "Wed, 30 Apr 2025 06:14:16 GMT"} })
32842025-04-30T06:14:17.838ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=0
32852025-04-30T06:14:17.844ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4c87aac4-665d-4ca0-bf95-73f6b3d9a4fd", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:16 GMT"} })
32862025-04-30T06:14:17.844ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=1
32872025-04-30T06:14:17.851ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e9c6bd2e-738e-4334-928f-fb867a570bad", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
32882025-04-30T06:14:17.852ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=0
32892025-04-30T06:14:17.856ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "52436405-f447-4aef-b8c6-81db828dfa1d", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
32902025-04-30T06:14:17.856ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=1
32912025-04-30T06:14:17.859ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c63a1539-bf41-47ba-9570-75bc5f5ce956", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
32922025-04-30T06:14:17.859ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=0
32932025-04-30T06:14:17.865ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "593435b3-a0f0-4028-bdfb-8062ebfb7dda", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
32942025-04-30T06:14:17.865ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=1
32952025-04-30T06:14:17.868ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a95661d0-5ab6-4f57-819e-5b6b77ce5763", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
32962025-04-30T06:14:17.869ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cmpa
32972025-04-30T06:14:17.880ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "96362a87-9b4e-432b-b227-9da8e7ddc135", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
32982025-04-30T06:14:17.881ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
32992025-04-30T06:14:17.883ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a2a1c01f-fc5b-4791-b91f-def2bd832851", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
33002025-04-30T06:14:17.883ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
33012025-04-30T06:14:17.884ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9f9a35d7-d6fe-4fb1-9808-8ef8d8399979", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
33022025-04-30T06:14:17.884ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
33032025-04-30T06:14:17.887ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a667cb5-674c-4a8c-88b6-1bd584d949ab", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
33042025-04-30T06:14:17.887ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1
33052025-04-30T06:14:17.888ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c1d37f8a-e6ef-4caf-ab99-272dfe566af2", "content-length": "723", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
33062025-04-30T06:14:17.888ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=0
33072025-04-30T06:14:17.893ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7193daad-1391-4f94-9e7f-a31af1609530", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
33082025-04-30T06:14:17.894ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=1
33092025-04-30T06:14:17.899ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "26b13bd3-5b56-4c75-8fdf-49e5f89fae62", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
33102025-04-30T06:14:17.899ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=0
33112025-04-30T06:14:17.903ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ec3223f3-683e-48a5-af93-7e211f73ba57", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
33122025-04-30T06:14:17.911ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=1
33132025-04-30T06:14:17.917ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "297ae921-10d6-46f7-bc5f-06eeae681945", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
33142025-04-30T06:14:17.917ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=0
33152025-04-30T06:14:17.919ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a18ed3bb-10c1-4da0-822d-bb7458e1b67d", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
33162025-04-30T06:14:17.920ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=1
33172025-04-30T06:14:17.924ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e7931170-9d2b-48b9-b595-b73022dfa59c", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
33182025-04-30T06:14:17.924ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cmpa
33192025-04-30T06:14:17.926ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "990b7bc5-5f50-4c69-aafd-c36061ca6557", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
33202025-04-30T06:14:17.926ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
33212025-04-30T06:14:17.928ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1ad626ff-be74-4f67-af9c-eb3ebbc54d62", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
33222025-04-30T06:14:17.928ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
33232025-04-30T06:14:17.929ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5d72ee8c-6b10-4d78-93aa-317e2246cf12", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
33242025-04-30T06:14:17.929ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
33252025-04-30T06:14:17.930ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3968ee82-72f8-4049-a378-6d1988c50009", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
33262025-04-30T06:14:17.930ZTRACtest_basic: authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet
33272025-04-30T06:14:17.930ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
33282025-04-30T06:14:17.931ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
33292025-04-30T06:14:17.931ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
33302025-04-30T06:14:17.932ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
33312025-04-30T06:14:17.939ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33322025-04-30T06:14:17.954ZDEBGtest_basic: authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet result = Ok(())
33332025-04-30T06:14:17.954ZTRACtest_basic: authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet
33342025-04-30T06:14:17.954ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
33352025-04-30T06:14:17.954ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
33362025-04-30T06:14:17.954ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
33372025-04-30T06:14:17.955ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
33382025-04-30T06:14:17.960ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33392025-04-30T06:14:17.965ZDEBGtest_basic: authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet result = Ok(())
33402025-04-30T06:14:17.965ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
33412025-04-30T06:14:17.965ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
33422025-04-30T06:14:17.965ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
33432025-04-30T06:14:18.076ZDEBGtest_basic: begin collection from Sled Agent sled_agent_url = http://[::1]:42420
33442025-04-30T06:14:18.076ZDEBGtest_basic: client request SledAgent = http://[::1]:42420 body = None method = GET uri = http://[::1]:42420/inventory
33452025-04-30T06:14:18.077ZINFOtest_basic (dropshot): accepted connection local_addr = [::1]:42420 remote_addr = [::1]:58373 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
33462025-04-30T06:14:18.079ZTRACtest_basic (dropshot): incoming request local_addr = [::1]:42420 method = GET remote_addr = [::1]:58373 req_id = 43d5b6f1-67b0-4009-a14f-bb91440175d2 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
33472025-04-30T06:14:18.081ZINFOtest_basic (dropshot): request completed latency_us = 2340 local_addr = [::1]:42420 method = GET remote_addr = [::1]:58373 req_id = 43d5b6f1-67b0-4009-a14f-bb91440175d2 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
33482025-04-30T06:14:18.083ZDEBGtest_basic: client response SledAgent = http://[::1]:42420 result = Ok(Response { url: "http://[::1]:42420/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "43d5b6f1-67b0-4009-a14f-bb91440175d2", "content-length": "2769", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
33492025-04-30T06:14:18.085ZDEBGtest_basic: begin collecting all keepers nkeeper_admin_clients = 0
33502025-04-30T06:14:18.085ZDEBGtest_basic: end collecting all keepers nkeeper_admin_clients = 0
33512025-04-30T06:14:18.086ZDEBGtest_basic: finished collection
33522025-04-30T06:14:18.092ZTRACtest_basic: authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory
33532025-04-30T06:14:18.092ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
33542025-04-30T06:14:18.092ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
33552025-04-30T06:14:18.092ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
33562025-04-30T06:14:18.093ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
33572025-04-30T06:14:18.102ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33582025-04-30T06:14:18.112ZDEBGtest_basic: authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory result = Ok(())
33592025-04-30T06:14:18.113ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
33602025-04-30T06:14:18.113ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
33612025-04-30T06:14:18.114ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
33622025-04-30T06:14:18.256ZWARNtest_basic (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
33632025-04-30T06:14:18.256ZWARNtest_basic: inventory collection failed error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
33642025-04-30T06:14:18.261ZTRACtest_basic: authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory
33652025-04-30T06:14:18.261ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
33662025-04-30T06:14:18.261ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
33672025-04-30T06:14:18.261ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
33682025-04-30T06:14:18.262ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
33692025-04-30T06:14:18.265ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33702025-04-30T06:14:18.269ZDEBGtest_basic: authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory result = Ok(())
33712025-04-30T06:14:18.269ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
33722025-04-30T06:14:18.269ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
33732025-04-30T06:14:18.269ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
33742025-04-30T06:14:18.274ZDEBGtest_basic: inventory_prune_one: nothing eligible for removal (too few) candidates = [(efcb0ee5-cafa-4692-9b75-fcee4732bdca, 0)]
33752025-04-30T06:14:18.274ZTRACtest_basic: lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
33762025-04-30T06:14:18.274ZTRACtest_basic (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: ce 84 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:51488 req_id = abccd3c3-eee5-4f57-97be-ad70ca40112e
33772025-04-30T06:14:18.274ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 52868,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51488 req_id = abccd3c3-eee5-4f57-97be-ad70ca40112e
33782025-04-30T06:14:18.274ZDEBGtest_basic (store): query key key = _mgs._tcp
33792025-04-30T06:14:18.275ZDEBGtest_basic (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
33802025-04-30T06:14:18.275ZDEBGtest_basic (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:51488 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 37822, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = abccd3c3-eee5-4f57-97be-ad70ca40112e
33812025-04-30T06:14:18.275ZDEBGtest_basic: lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 37822, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2523, tv_nsec: 74745527 } })
33822025-04-30T06:14:18.276ZTRACtest_basic: lookup_service_targets: looking up SRV target
33832025-04-30T06:14:18.276ZTRACtest_basic (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: d9 20 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e . ..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:60797 req_id = bc6c388b-6d1f-4099-98a9-c8e258c12437
33842025-04-30T06:14:18.276ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 55584,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60797 req_id = bc6c388b-6d1f-4099-98a9-c8e258c12437
33852025-04-30T06:14:18.276ZDEBGtest_basic (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
33862025-04-30T06:14:18.276ZDEBGtest_basic (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:60797 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = bc6c388b-6d1f-4099-98a9-c8e258c12437
33872025-04-30T06:14:18.348ZTRACtest_basic: lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
33882025-04-30T06:14:18.350ZTRACtest_basic (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: ad 6d 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c .m..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:55609 req_id = 282e57d8-4ec8-46b5-8496-5ac0399a5131
33892025-04-30T06:14:18.350ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 44397,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:55609 req_id = 282e57d8-4ec8-46b5-8496-5ac0399a5131
33902025-04-30T06:14:18.350ZDEBGtest_basic (store): query key key = _clickhouse-admin-keeper._tcp
33912025-04-30T06:14:18.351ZERROtest_basic (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 44397, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:55609 req_id = 282e57d8-4ec8-46b5-8496-5ac0399a5131
33922025-04-30T06:14:18.351ZDEBGtest_basic: begin collection
33932025-04-30T06:14:18.351ZDEBGtest_basic: begin collection from MGS mgs_url = http://[::1]:37822
33942025-04-30T06:14:18.351ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/ignition
33952025-04-30T06:14:18.353ZDEBGtest_basic (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
33962025-04-30T06:14:18.354ZDEBGtest_basic (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
33972025-04-30T06:14:18.355ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "62e46ac0-797d-4267-8ef3-c2e8707b3bb2", "content-length": "835", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
33982025-04-30T06:14:18.355ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0
33992025-04-30T06:14:18.361ZTRACtest_basic (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
34002025-04-30T06:14:18.364ZDEBGtest_basic (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
34012025-04-30T06:14:18.364ZTRACtest_basic (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
34022025-04-30T06:14:18.365ZTRACtest_basic (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
34032025-04-30T06:14:18.371ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "31c0e03a-6dd7-46c8-bec8-6a93af85f545", "content-length": "734", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
34042025-04-30T06:14:18.371ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=0
34052025-04-30T06:14:18.379ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b165fdaf-6013-4167-ad26-2bb571826216", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
34062025-04-30T06:14:18.379ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=1
34072025-04-30T06:14:18.383ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f6222969-058f-48d9-b0cb-06b7def4e60a", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
34082025-04-30T06:14:18.383ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=0
34092025-04-30T06:14:18.386ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "02b854e2-2297-4de0-ad02-7ddcc5e2127f", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
34102025-04-30T06:14:18.386ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=1
34112025-04-30T06:14:18.391ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "14f96acb-a051-4a91-9916-2f1e5dada392", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
34122025-04-30T06:14:18.391ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=0
34132025-04-30T06:14:18.395ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d1e30370-06ba-4abe-addb-ee675b248a0f", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
34142025-04-30T06:14:18.395ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=1
34152025-04-30T06:14:18.399ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e5601308-ad9b-4b39-abfe-eef9919c87a8", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
34162025-04-30T06:14:18.400ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cmpa
34172025-04-30T06:14:18.401ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7f456e9c-005a-484b-8fb4-c67e8241e7b0", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
34182025-04-30T06:14:18.401ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
34192025-04-30T06:14:18.402ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fb78c2bb-38ee-4f54-8127-e5143c6aef44", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
34202025-04-30T06:14:18.402ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
34212025-04-30T06:14:18.403ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f41a5274-ac51-4db3-bbf1-fc6363148b3d", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
34222025-04-30T06:14:18.404ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
34232025-04-30T06:14:18.410ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a1d6e2c1-aad2-426d-88fc-8c9fd9d61caf", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
34242025-04-30T06:14:18.410ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1
34252025-04-30T06:14:18.412ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "eee89fb0-37e9-405b-99c9-4204ad324dd6", "content-length": "734", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
34262025-04-30T06:14:18.412ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=0
34272025-04-30T06:14:18.420ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e4b7220c-6120-4091-99e5-1c23b86c52c7", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
34282025-04-30T06:14:18.420ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=1
34292025-04-30T06:14:18.430ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d835d158-947a-487c-82e8-03b20490a037", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
34302025-04-30T06:14:18.430ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=0
34312025-04-30T06:14:18.438ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "37254edb-27c4-4023-bec7-dcf75bce1ee0", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
34322025-04-30T06:14:18.439ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=1
34332025-04-30T06:14:18.444ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "607b042e-de16-46ac-a19b-7b09699af592", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
34342025-04-30T06:14:18.445ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=0
34352025-04-30T06:14:18.450ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ab512aed-5995-4a5a-a7b5-6dfaf11974fb", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
34362025-04-30T06:14:18.452ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=1
34372025-04-30T06:14:18.454ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "db923e6f-d53e-4207-85d1-79f224de9279", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
34382025-04-30T06:14:18.454ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cmpa
34392025-04-30T06:14:18.456ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "261009f5-999b-4190-b79e-fcadb3fa4b43", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
34402025-04-30T06:14:18.457ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
34412025-04-30T06:14:18.459ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "613b4369-7ddc-430d-ba78-5c4304dc05c4", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
34422025-04-30T06:14:18.460ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
34432025-04-30T06:14:18.464ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "df142ddb-1a3e-4695-925f-714b353df1a5", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
34442025-04-30T06:14:18.465ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
34452025-04-30T06:14:18.468ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a5676f51-f53a-42a0-bf52-d3d2aadd6262", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
34462025-04-30T06:14:18.468ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0
34472025-04-30T06:14:18.470ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "63cfc8d1-336d-4451-83a1-492e1a90f050", "content-length": "723", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
34482025-04-30T06:14:18.470ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=0
34492025-04-30T06:14:18.476ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5abfe8d0-d607-462d-bc75-baa5c8d88f54", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
34502025-04-30T06:14:18.476ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=1
34512025-04-30T06:14:18.480ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "57b53469-cbdd-4164-8c03-70ad62f4cd6c", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
34522025-04-30T06:14:18.481ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=0
34532025-04-30T06:14:18.487ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "01e46d5e-dc57-4dfe-9c48-632a17a44ee7", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
34542025-04-30T06:14:18.488ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=1
34552025-04-30T06:14:18.493ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5c21509c-25d2-47d1-b742-6c0b2535da95", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
34562025-04-30T06:14:18.493ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=0
34572025-04-30T06:14:18.503ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e03142a7-8780-4dbb-b28b-a44b8e9469c4", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
34582025-04-30T06:14:18.510ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=1
34592025-04-30T06:14:18.527ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "640cd713-673b-4cf5-9e85-f9ab75eac1a0", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
34602025-04-30T06:14:18.527ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cmpa
34612025-04-30T06:14:18.529ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "df9cbebf-5d23-4868-b0bd-df5e0a7753ff", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
34622025-04-30T06:14:18.529ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
34632025-04-30T06:14:18.532ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0d0a9d5d-37d7-478b-84dc-a8354c175f95", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
34642025-04-30T06:14:18.533ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
34652025-04-30T06:14:18.546ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8e932395-1c52-43b4-b8bd-d50554f94923", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
34662025-04-30T06:14:18.547ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
34672025-04-30T06:14:18.549ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "406be3d4-ba50-4f90-a691-7bae1b6e2f10", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
34682025-04-30T06:14:18.549ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1
34692025-04-30T06:14:18.553ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "705e34ce-c4e2-41c2-8246-86ed861e4c56", "content-length": "723", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
34702025-04-30T06:14:18.553ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=0
34712025-04-30T06:14:18.559ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f40a1741-10ed-41ca-b90e-a1414e1aefa7", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
34722025-04-30T06:14:18.559ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=1
34732025-04-30T06:14:18.570ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "62a93e76-3c48-4f16-baf1-de5387df60d2", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
34742025-04-30T06:14:18.571ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=0
34752025-04-30T06:14:18.575ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9c137494-1729-4b02-8945-3b1c228d2143", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
34762025-04-30T06:14:18.575ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=1
34772025-04-30T06:14:18.584ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a4d0e446-e38c-4027-ba64-67da4dfab4db", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
34782025-04-30T06:14:18.585ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=0
34792025-04-30T06:14:18.591ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "583b01e6-8a3e-4486-ac0d-e676a37d775d", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
34802025-04-30T06:14:18.592ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=1
34812025-04-30T06:14:18.619ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "eb271660-23ec-403a-b0b9-d2074e56c0d7", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
34822025-04-30T06:14:18.619ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cmpa
34832025-04-30T06:14:18.625ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "861305ef-7ee8-4d2f-a0ca-1a5a91c7c72a", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
34842025-04-30T06:14:18.628ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
34852025-04-30T06:14:18.629ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6e9ba32e-93f4-4210-8ed0-2259a7d9c9cb", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
34862025-04-30T06:14:18.629ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
34872025-04-30T06:14:18.632ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a79ca20-b7fa-4304-b1e0-89ffdc57cb1c", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
34882025-04-30T06:14:18.632ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
34892025-04-30T06:14:18.633ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "771a9bdd-dcbf-499d-9c66-7cdd9e21080d", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
34902025-04-30T06:14:18.633ZTRACtest_basic: authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet
34912025-04-30T06:14:18.633ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
34922025-04-30T06:14:18.633ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
34932025-04-30T06:14:18.633ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
34942025-04-30T06:14:18.635ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
34952025-04-30T06:14:18.639ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34962025-04-30T06:14:18.644ZDEBGtest_basic: authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet result = Ok(())
34972025-04-30T06:14:18.644ZTRACtest_basic: authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet
34982025-04-30T06:14:18.645ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
34992025-04-30T06:14:18.645ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
35002025-04-30T06:14:18.645ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
35012025-04-30T06:14:18.646ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
35022025-04-30T06:14:18.650ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35032025-04-30T06:14:18.655ZDEBGtest_basic: authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet result = Ok(())
35042025-04-30T06:14:18.655ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
35052025-04-30T06:14:18.655ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
35062025-04-30T06:14:18.656ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
35072025-04-30T06:14:18.764ZDEBGtest_basic: begin collection from Sled Agent sled_agent_url = http://[::1]:42420
35082025-04-30T06:14:18.764ZDEBGtest_basic: client request SledAgent = http://[::1]:42420 body = None method = GET uri = http://[::1]:42420/inventory
35092025-04-30T06:14:18.765ZINFOtest_basic (dropshot): accepted connection local_addr = [::1]:42420 remote_addr = [::1]:55735 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
35102025-04-30T06:14:18.765ZDEBGtest_basic (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
35112025-04-30T06:14:18.765ZDEBGtest_basic (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
35122025-04-30T06:14:18.765ZDEBGtest_basic (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812
35132025-04-30T06:14:18.766ZDEBGtest_basic (clickhouse-client): retrieving timeseries schema from database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812
35142025-04-30T06:14:18.766ZTRACtest_basic (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 sql = SELECT * FROM oximeter.timeseries_schema FORMAT Native;
35152025-04-30T06:14:18.768ZTRACtest_basic (dropshot): incoming request local_addr = [::1]:42420 method = GET remote_addr = [::1]:55735 req_id = e27ea683-9430-4d5b-a6bb-d7bc9abe12d3 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
35162025-04-30T06:14:18.769ZINFOtest_basic (dropshot): request completed latency_us = 834 local_addr = [::1]:42420 method = GET remote_addr = [::1]:55735 req_id = e27ea683-9430-4d5b-a6bb-d7bc9abe12d3 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
35172025-04-30T06:14:18.773ZTRACtest_basic (clickhouse-client): no new timeseries schema in database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812
35182025-04-30T06:14:18.773ZDEBGtest_basic (clickhouse-client): new timeseries schema collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 schema = TimeseriesSchema { timeseries_name: TimeseriesName("integration_target:integration_metric"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "metric_name", field_type: String, source: Metric, description: "" }, FieldSchema { name: "target_name", field_type: String, source: Target, description: "" }}, datum_type: I64, version: 1, authz_scope: Fleet, units: Count, created: 2025-04-30T06:14:18.766803552Z } timeseries_name = integration_target:integration_metric
35192025-04-30T06:14:18.773ZDEBGtest_basic (clickhouse-client): inserting 1 new timeseries schema collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812
35202025-04-30T06:14:18.774ZTRACtest_basic (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_columns = 6 n_rows = 1 sql = INSERT INTO oximeter.timeseries_schema FORMAT Native
35212025-04-30T06:14:18.775ZDEBGtest_basic: client response SledAgent = http://[::1]:42420 result = Ok(Response { url: "http://[::1]:42420/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "e27ea683-9430-4d5b-a6bb-d7bc9abe12d3", "content-length": "2769", "date": "Wed, 30 Apr 2025 06:14:17 GMT"} })
35222025-04-30T06:14:18.784ZDEBGtest_basic: begin collecting all keepers nkeeper_admin_clients = 0
35232025-04-30T06:14:18.785ZDEBGtest_basic: end collecting all keepers nkeeper_admin_clients = 0
35242025-04-30T06:14:18.785ZDEBGtest_basic: finished collection
35252025-04-30T06:14:18.791ZTRACtest_basic: authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory
35262025-04-30T06:14:18.792ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
35272025-04-30T06:14:18.792ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
35282025-04-30T06:14:18.792ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
35292025-04-30T06:14:18.794ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
35302025-04-30T06:14:18.802ZTRACtest_basic (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
35312025-04-30T06:14:18.806ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35322025-04-30T06:14:18.814ZDEBGtest_basic: authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory result = Ok(())
35332025-04-30T06:14:18.814ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
35342025-04-30T06:14:18.814ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
35352025-04-30T06:14:18.815ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
35362025-04-30T06:14:18.816ZDEBGtest_basic (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_rows = 2 table_name = fields_string
35372025-04-30T06:14:18.816ZTRACtest_basic (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
35382025-04-30T06:14:18.823ZDEBGtest_basic (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_rows = 1 table_name = measurements_i64
35392025-04-30T06:14:18.823ZTRACtest_basic (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
35402025-04-30T06:14:18.878ZWARNtest_basic (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
35412025-04-30T06:14:18.892ZWARNtest_basic (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
35422025-04-30T06:14:18.892ZWARNtest_basic: inventory collection failed error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
35432025-04-30T06:14:18.898ZTRACtest_basic: authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory
35442025-04-30T06:14:18.898ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
35452025-04-30T06:14:18.898ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
35462025-04-30T06:14:18.898ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
35472025-04-30T06:14:18.899ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
35482025-04-30T06:14:18.903ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35492025-04-30T06:14:18.913ZDEBGtest_basic: authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory result = Ok(())
35502025-04-30T06:14:18.913ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
35512025-04-30T06:14:18.913ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
35522025-04-30T06:14:18.914ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
35532025-04-30T06:14:18.918ZDEBGtest_basic: inventory_prune_one: nothing eligible for removal (too few) candidates = [(efcb0ee5-cafa-4692-9b75-fcee4732bdca, 0)]
35542025-04-30T06:14:18.918ZTRACtest_basic: lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
35552025-04-30T06:14:18.918ZTRACtest_basic (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: f0 c7 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:63414 req_id = 8dfb235a-300f-4746-95e2-6801214235f9
35562025-04-30T06:14:18.919ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 61639,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63414 req_id = 8dfb235a-300f-4746-95e2-6801214235f9
35572025-04-30T06:14:18.919ZDEBGtest_basic (store): query key key = _mgs._tcp
35582025-04-30T06:14:18.919ZDEBGtest_basic (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
35592025-04-30T06:14:18.919ZDEBGtest_basic (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:63414 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 37822, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 8dfb235a-300f-4746-95e2-6801214235f9
35602025-04-30T06:14:18.920ZDEBGtest_basic: lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 37822, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2523, tv_nsec: 718996324 } })
35612025-04-30T06:14:18.920ZTRACtest_basic: lookup_service_targets: looking up SRV target
35622025-04-30T06:14:18.920ZTRACtest_basic (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 6a 19 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e j...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:50086 req_id = f126cb3a-6058-4f1d-ba91-df056676fa52
35632025-04-30T06:14:18.920ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 27161,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:50086 req_id = f126cb3a-6058-4f1d-ba91-df056676fa52
35642025-04-30T06:14:18.921ZDEBGtest_basic (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
35652025-04-30T06:14:18.921ZDEBGtest_basic (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:50086 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = f126cb3a-6058-4f1d-ba91-df056676fa52
35662025-04-30T06:14:19.005ZTRACtest_basic: lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
35672025-04-30T06:14:19.006ZINFOtest_basic (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
35682025-04-30T06:14:19.006ZDEBGtest_basic (oximeter): client request body = None method = GET uri = http://127.0.0.1:34211/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
35692025-04-30T06:14:19.007ZTRACtest_basic (dropshot_internal): incoming request local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 8629a05a-6230-4fc6-854d-6a6cd2dccec8 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
35702025-04-30T06:14:19.007ZTRACtest_basic (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 8629a05a-6230-4fc6-854d-6a6cd2dccec8 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
35712025-04-30T06:14:19.007ZDEBGtest_basic (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 8629a05a-6230-4fc6-854d-6a6cd2dccec8 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
35722025-04-30T06:14:19.009ZDEBGtest_basic (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 8629a05a-6230-4fc6-854d-6a6cd2dccec8 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
35732025-04-30T06:14:19.010ZTRACtest_basic (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 69 fa 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c i............_cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:50027 req_id = a23a7402-c109-4c83-9a00-aa3b3628e336
35742025-04-30T06:14:19.010ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 27130,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:50027 req_id = a23a7402-c109-4c83-9a00-aa3b3628e336
35752025-04-30T06:14:19.010ZDEBGtest_basic (store): query key key = _clickhouse-admin-keeper._tcp
35762025-04-30T06:14:19.010ZERROtest_basic (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 27130, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:50027 req_id = a23a7402-c109-4c83-9a00-aa3b3628e336
35772025-04-30T06:14:19.020ZDEBGtest_basic: begin collection
35782025-04-30T06:14:19.020ZDEBGtest_basic: begin collection from MGS mgs_url = http://[::1]:37822
35792025-04-30T06:14:19.022ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/ignition
35802025-04-30T06:14:19.023ZINFOtest_basic (dropshot_internal): request completed latency_us = 15818 local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 8629a05a-6230-4fc6-854d-6a6cd2dccec8 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
35812025-04-30T06:14:19.023ZDEBGtest_basic (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:34211/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "8629a05a-6230-4fc6-854d-6a6cd2dccec8", "content-length": "405", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
35822025-04-30T06:14:19.024ZDEBGtest_basic (oximeter): client request body = None method = GET uri = http://127.0.0.1:34211/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
35832025-04-30T06:14:19.024ZTRACtest_basic (dropshot_internal): incoming request local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 7c7b6249-7f85-4823-a121-b0f639c4fb37 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
35842025-04-30T06:14:19.025ZTRACtest_basic (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 7c7b6249-7f85-4823-a121-b0f639c4fb37 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
35852025-04-30T06:14:19.025ZDEBGtest_basic (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 7c7b6249-7f85-4823-a121-b0f639c4fb37 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
35862025-04-30T06:14:19.026ZDEBGtest_basic (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 7c7b6249-7f85-4823-a121-b0f639c4fb37 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
35872025-04-30T06:14:19.029ZINFOtest_basic (dropshot_internal): request completed latency_us = 4975 local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 7c7b6249-7f85-4823-a121-b0f639c4fb37 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
35882025-04-30T06:14:19.030ZDEBGtest_basic (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:34211/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "7c7b6249-7f85-4823-a121-b0f639c4fb37", "content-length": "29", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
35892025-04-30T06:14:19.034ZTRACtest_basic (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 088ac96c-981b-4054-bea6-d8ab23ae1366
35902025-04-30T06:14:19.034ZTRACtest_basic (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
35912025-04-30T06:14:19.034ZINFOtest_basic (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
35922025-04-30T06:14:19.034ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "1d3663fa-d46d-47a6-af4c-3fb064acacb2", "content-length": "835", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
35932025-04-30T06:14:19.035ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0
35942025-04-30T06:14:19.037ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a8b80a48-337a-4fbe-b976-59d6967416ad", "content-length": "734", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
35952025-04-30T06:14:19.037ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=0
35962025-04-30T06:14:19.039ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "84e94963-3484-4d42-8ab0-c46a0dcf734b", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
35972025-04-30T06:14:19.039ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=1
35982025-04-30T06:14:19.041ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8ab0a0b7-0513-4c2d-a6a0-0dec482278cd", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
35992025-04-30T06:14:19.041ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=0
36002025-04-30T06:14:19.043ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4f1d5b0a-fa21-4847-8a89-fce709141d54", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
36012025-04-30T06:14:19.043ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=1
36022025-04-30T06:14:19.049ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "929a3594-9d9a-42d4-a84e-ddeaf227c9ec", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
36032025-04-30T06:14:19.050ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=0
36042025-04-30T06:14:19.060ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e900d7c2-4408-4a3a-9341-2de4b2110038", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
36052025-04-30T06:14:19.061ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=1
36062025-04-30T06:14:19.069ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4c7dde7b-df5e-4353-ba68-218b75428e2e", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
36072025-04-30T06:14:19.070ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cmpa
36082025-04-30T06:14:19.074ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b41c3a7f-749a-4ac2-8cac-60c8dd9cc4bb", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
36092025-04-30T06:14:19.075ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
36102025-04-30T06:14:19.077ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b43a7be6-8add-4823-bf17-ebaaac99c132", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
36112025-04-30T06:14:19.077ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
36122025-04-30T06:14:19.087ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "43a54c9c-d797-4b41-8352-77bc564dbb9c", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
36132025-04-30T06:14:19.087ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
36142025-04-30T06:14:19.106ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "43ef2a70-5be7-4c55-b2c0-99f5ae95b56e", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
36152025-04-30T06:14:19.106ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1
36162025-04-30T06:14:19.137ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f638027a-d9bd-4cde-897a-3df2d6079b82", "content-length": "734", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
36172025-04-30T06:14:19.137ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=0
36182025-04-30T06:14:19.148ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f5a6f99e-ba18-4a0f-b7aa-853649ca9f22", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
36192025-04-30T06:14:19.148ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=1
36202025-04-30T06:14:19.152ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "132f3fbd-c7ad-4cf3-8c73-47991a5f8027", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
36212025-04-30T06:14:19.153ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=0
36222025-04-30T06:14:19.161ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "314d9818-747f-4d89-a266-81ba6b5603eb", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
36232025-04-30T06:14:19.161ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=1
36242025-04-30T06:14:19.164ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5d08a748-a1a4-4135-8ffc-e037d1fa7719", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
36252025-04-30T06:14:19.165ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=0
36262025-04-30T06:14:19.183ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "94f986cb-5cce-4ad4-9b4d-a9b55a825b08", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
36272025-04-30T06:14:19.184ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=1
36282025-04-30T06:14:19.191ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a82bea31-ad44-4d92-9c37-a07dc1dbf702", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
36292025-04-30T06:14:19.191ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cmpa
36302025-04-30T06:14:19.192ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a3e6066f-dc7a-4123-ae49-e357ba8720ab", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
36312025-04-30T06:14:19.192ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
36322025-04-30T06:14:19.194ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "16bbe75e-f11a-44fb-bee9-63ecd175bd18", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
36332025-04-30T06:14:19.194ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
36342025-04-30T06:14:19.196ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b8d79952-dfe8-44d5-86eb-26cefcba9a31", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
36352025-04-30T06:14:19.197ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
36362025-04-30T06:14:19.199ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "abf5bc23-b152-4191-afd6-3c907d783095", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
36372025-04-30T06:14:19.199ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0
36382025-04-30T06:14:19.201ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "662318b1-b6c3-46fb-831f-bb1bddc4f711", "content-length": "723", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
36392025-04-30T06:14:19.201ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=0
36402025-04-30T06:14:19.207ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9b960ec1-5a54-4bbd-ab4d-7614383070fe", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
36412025-04-30T06:14:19.215ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=1
36422025-04-30T06:14:19.221ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8dfe8a6a-084b-46d7-a5aa-9796a2b5f5fd", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
36432025-04-30T06:14:19.221ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=0
36442025-04-30T06:14:19.226ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7253511d-595b-43cd-9f66-328da9bc532c", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
36452025-04-30T06:14:19.226ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=1
36462025-04-30T06:14:19.236ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d01b5157-cc4e-4500-a637-aa99d8fd9b13", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
36472025-04-30T06:14:19.237ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=0
36482025-04-30T06:14:19.240ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "84f1662f-766c-4213-aecd-4987b55f0bbd", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
36492025-04-30T06:14:19.240ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=1
36502025-04-30T06:14:19.247ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5696b61b-4009-4a19-baeb-90485f931b0c", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
36512025-04-30T06:14:19.250ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cmpa
36522025-04-30T06:14:19.253ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e7bfd47c-b4d1-4688-b52b-2b31b06a400d", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
36532025-04-30T06:14:19.253ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
36542025-04-30T06:14:19.257ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6a29ce0d-0449-4d69-9598-31bdbaf6405b", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
36552025-04-30T06:14:19.257ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
36562025-04-30T06:14:19.261ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f91d7ef0-60cc-40a6-8151-15b535c0229e", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
36572025-04-30T06:14:19.262ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
36582025-04-30T06:14:19.268ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ee363232-0c0b-4db1-8d93-6011328babf0", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
36592025-04-30T06:14:19.273ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1
36602025-04-30T06:14:19.276ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bccf9555-a931-4f1c-b6e4-b1de6200e139", "content-length": "723", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
36612025-04-30T06:14:19.277ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=0
36622025-04-30T06:14:19.279ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e5ddca0a-36f7-460c-97fc-8cf432c2d629", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
36632025-04-30T06:14:19.280ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=1
36642025-04-30T06:14:19.286ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "742d606b-8899-40f9-a39f-93fac7604ade", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
36652025-04-30T06:14:19.286ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=0
36662025-04-30T06:14:19.297ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7361b1a2-972b-4218-b0b8-5b1c108264c1", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
36672025-04-30T06:14:19.298ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=1
36682025-04-30T06:14:19.302ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c14947ac-9ab0-406a-97b8-53d2a98ca19c", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
36692025-04-30T06:14:19.302ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=0
36702025-04-30T06:14:19.308ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "915a267a-60ba-4af6-bd14-8a837c260847", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
36712025-04-30T06:14:19.308ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=1
36722025-04-30T06:14:19.317ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "47baab6d-79ed-4462-94b7-5b3eb06ebfd4", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
36732025-04-30T06:14:19.317ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cmpa
36742025-04-30T06:14:19.319ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "39472378-4fa7-4444-beaf-79d2279351d1", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
36752025-04-30T06:14:19.319ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
36762025-04-30T06:14:19.321ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2c36cb45-6831-43f7-afff-6f916a31bd00", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
36772025-04-30T06:14:19.321ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
36782025-04-30T06:14:19.323ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1991e334-2fba-4086-ac7d-b769556cc13f", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
36792025-04-30T06:14:19.324ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
36802025-04-30T06:14:19.327ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "23dd4092-ca37-4394-919e-e8293ffe9798", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
36812025-04-30T06:14:19.327ZTRACtest_basic: authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet
36822025-04-30T06:14:19.327ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
36832025-04-30T06:14:19.327ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
36842025-04-30T06:14:19.327ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
36852025-04-30T06:14:19.328ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
36862025-04-30T06:14:19.337ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36872025-04-30T06:14:19.348ZDEBGtest_basic: authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet result = Ok(())
36882025-04-30T06:14:19.348ZTRACtest_basic: authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet
36892025-04-30T06:14:19.348ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
36902025-04-30T06:14:19.348ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
36912025-04-30T06:14:19.348ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
36922025-04-30T06:14:19.349ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
36932025-04-30T06:14:19.354ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36942025-04-30T06:14:19.368ZDEBGtest_basic: authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet result = Ok(())
36952025-04-30T06:14:19.368ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
36962025-04-30T06:14:19.368ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
36972025-04-30T06:14:19.371ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
36982025-04-30T06:14:19.371ZDEBGtest_basic (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
36992025-04-30T06:14:19.372ZDEBGtest_basic (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
37002025-04-30T06:14:19.556ZDEBGtest_basic: begin collection from Sled Agent sled_agent_url = http://[::1]:42420
37012025-04-30T06:14:19.556ZDEBGtest_basic: client request SledAgent = http://[::1]:42420 body = None method = GET uri = http://[::1]:42420/inventory
37022025-04-30T06:14:19.558ZTRACtest_basic (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
37032025-04-30T06:14:19.563ZDEBGtest_basic (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
37042025-04-30T06:14:19.563ZTRACtest_basic (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
37052025-04-30T06:14:19.563ZTRACtest_basic (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
37062025-04-30T06:14:19.564ZINFOtest_basic (dropshot): accepted connection local_addr = [::1]:42420 remote_addr = [::1]:47368 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
37072025-04-30T06:14:19.565ZTRACtest_basic (dropshot): incoming request local_addr = [::1]:42420 method = GET remote_addr = [::1]:47368 req_id = cf111eaf-5cfd-4497-9e5b-1ddd92736bd9 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
37082025-04-30T06:14:19.566ZINFOtest_basic (dropshot): request completed latency_us = 950 local_addr = [::1]:42420 method = GET remote_addr = [::1]:47368 req_id = cf111eaf-5cfd-4497-9e5b-1ddd92736bd9 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
37092025-04-30T06:14:19.567ZDEBGtest_basic: client response SledAgent = http://[::1]:42420 result = Ok(Response { url: "http://[::1]:42420/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "cf111eaf-5cfd-4497-9e5b-1ddd92736bd9", "content-length": "2769", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
37102025-04-30T06:14:19.571ZDEBGtest_basic: begin collecting all keepers nkeeper_admin_clients = 0
37112025-04-30T06:14:19.571ZDEBGtest_basic: end collecting all keepers nkeeper_admin_clients = 0
37122025-04-30T06:14:19.571ZDEBGtest_basic: finished collection
37132025-04-30T06:14:19.578ZTRACtest_basic: authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory
37142025-04-30T06:14:19.578ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
37152025-04-30T06:14:19.578ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
37162025-04-30T06:14:19.578ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
37172025-04-30T06:14:19.580ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
37182025-04-30T06:14:19.584ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37192025-04-30T06:14:19.592ZDEBGtest_basic: authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory result = Ok(())
37202025-04-30T06:14:19.593ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
37212025-04-30T06:14:19.593ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
37222025-04-30T06:14:19.594ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
37232025-04-30T06:14:19.642ZWARNtest_basic (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
37242025-04-30T06:14:19.643ZWARNtest_basic: inventory collection failed error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
37252025-04-30T06:14:19.648ZTRACtest_basic: authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory
37262025-04-30T06:14:19.648ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
37272025-04-30T06:14:19.648ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
37282025-04-30T06:14:19.648ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
37292025-04-30T06:14:19.649ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
37302025-04-30T06:14:19.653ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37312025-04-30T06:14:19.661ZDEBGtest_basic: authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory result = Ok(())
37322025-04-30T06:14:19.664ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
37332025-04-30T06:14:19.664ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
37342025-04-30T06:14:19.672ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
37352025-04-30T06:14:19.678ZDEBGtest_basic: inventory_prune_one: nothing eligible for removal (too few) candidates = [(efcb0ee5-cafa-4692-9b75-fcee4732bdca, 0)]
37362025-04-30T06:14:19.678ZTRACtest_basic: lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
37372025-04-30T06:14:19.679ZTRACtest_basic (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: b3 b3 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:50018 req_id = 7d0bc60d-97b1-433c-bd2e-de020e8d71b4
37382025-04-30T06:14:19.679ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 46003,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:50018 req_id = 7d0bc60d-97b1-433c-bd2e-de020e8d71b4
37392025-04-30T06:14:19.680ZDEBGtest_basic (store): query key key = _mgs._tcp
37402025-04-30T06:14:19.680ZDEBGtest_basic (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
37412025-04-30T06:14:19.680ZDEBGtest_basic (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:50018 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 37822, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 7d0bc60d-97b1-433c-bd2e-de020e8d71b4
37422025-04-30T06:14:19.681ZDEBGtest_basic: lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 37822, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2524, tv_nsec: 480291917 } })
37432025-04-30T06:14:19.681ZTRACtest_basic: lookup_service_targets: looking up SRV target
37442025-04-30T06:14:19.681ZTRACtest_basic (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: bb 2f 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ./..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:51198 req_id = 643fb526-e8b8-4656-9729-6a4fbded974c
37452025-04-30T06:14:19.682ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 47919,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51198 req_id = 643fb526-e8b8-4656-9729-6a4fbded974c
37462025-04-30T06:14:19.682ZDEBGtest_basic (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
37472025-04-30T06:14:19.683ZDEBGtest_basic (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:51198 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 643fb526-e8b8-4656-9729-6a4fbded974c
37482025-04-30T06:14:19.778ZTRACtest_basic: lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
37492025-04-30T06:14:19.779ZDEBGtest_basic (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
37502025-04-30T06:14:19.779ZDEBGtest_basic (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
37512025-04-30T06:14:19.781ZDEBGtest_basic (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812
37522025-04-30T06:14:19.782ZTRACtest_basic (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
37532025-04-30T06:14:19.783ZTRACtest_basic (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 11 f8 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:62665 req_id = 72fcf617-28b1-47cc-a295-f5689a2e889f
37542025-04-30T06:14:19.783ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 4600,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62665 req_id = 72fcf617-28b1-47cc-a295-f5689a2e889f
37552025-04-30T06:14:19.792ZDEBGtest_basic (store): query key key = _clickhouse-admin-keeper._tcp
37562025-04-30T06:14:19.792ZERROtest_basic (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 4600, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:62665 req_id = 72fcf617-28b1-47cc-a295-f5689a2e889f
37572025-04-30T06:14:19.793ZDEBGtest_basic: begin collection
37582025-04-30T06:14:19.793ZDEBGtest_basic: begin collection from MGS mgs_url = http://[::1]:37822
37592025-04-30T06:14:19.793ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/ignition
37602025-04-30T06:14:19.796ZDEBGtest_basic (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_rows = 2 table_name = fields_string
37612025-04-30T06:14:19.797ZTRACtest_basic (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
37622025-04-30T06:14:19.797ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "d4cdf88f-c18e-4a8c-96ca-9cd7ec37bf5d", "content-length": "835", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
37632025-04-30T06:14:19.798ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0
37642025-04-30T06:14:19.803ZDEBGtest_basic (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_rows = 1 table_name = measurements_i64
37652025-04-30T06:14:19.804ZTRACtest_basic (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
37662025-04-30T06:14:19.804ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c3e79754-a73e-43a0-aa46-f22c98c7475f", "content-length": "734", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
37672025-04-30T06:14:19.805ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=0
37682025-04-30T06:14:19.808ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e2147d99-bb06-4a07-bfa6-4d232656da6e", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
37692025-04-30T06:14:19.808ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=1
37702025-04-30T06:14:19.812ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "864541e5-6b05-428b-874c-adc5d9b48a5f", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
37712025-04-30T06:14:19.812ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=0
37722025-04-30T06:14:19.815ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cdcf126d-23b2-455c-88a4-16e6a8ba2742", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
37732025-04-30T06:14:19.815ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=1
37742025-04-30T06:14:19.817ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e4cb13f0-c2f1-4785-be77-5c3f218a6324", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
37752025-04-30T06:14:19.817ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=0
37762025-04-30T06:14:19.820ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "436a6126-b425-45f3-80df-9de358d05a4e", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
37772025-04-30T06:14:19.820ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=1
37782025-04-30T06:14:19.824ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f9447d09-cd36-4673-bfff-9b00e27929cd", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
37792025-04-30T06:14:19.824ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cmpa
37802025-04-30T06:14:19.826ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "dbd168ab-1836-460b-a7cf-2d3b323f1f9a", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
37812025-04-30T06:14:19.826ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
37822025-04-30T06:14:19.828ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b31e311f-5092-4086-8d06-f59102b27347", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
37832025-04-30T06:14:19.828ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
37842025-04-30T06:14:19.830ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a61bbe5e-9049-4701-aa95-fb6129376586", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
37852025-04-30T06:14:19.830ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
37862025-04-30T06:14:19.832ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f23aba43-d879-4b0f-ad2e-983efd274a23", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
37872025-04-30T06:14:19.832ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1
37882025-04-30T06:14:19.834ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6a38486d-ce4d-4b85-ae17-e53339ab02a5", "content-length": "734", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
37892025-04-30T06:14:19.834ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=0
37902025-04-30T06:14:19.836ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "42881073-a942-41df-b988-2232e501819e", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
37912025-04-30T06:14:19.836ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=1
37922025-04-30T06:14:19.839ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e7b2a23a-f1fb-46f1-9056-3e8ac93a9b3e", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
37932025-04-30T06:14:19.839ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=0
37942025-04-30T06:14:19.841ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "28e7a91f-0478-42f2-9951-0d0b68fae40d", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
37952025-04-30T06:14:19.841ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=1
37962025-04-30T06:14:19.844ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "157cb379-f6b8-494d-a4af-968b8d5ac758", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
37972025-04-30T06:14:19.844ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=0
37982025-04-30T06:14:19.851ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "aa6a950f-e74b-466b-be69-31084ba776d1", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
37992025-04-30T06:14:19.860ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=1
38002025-04-30T06:14:19.864ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f49f0404-e472-4db0-97a6-62333fb7edd0", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
38012025-04-30T06:14:19.864ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cmpa
38022025-04-30T06:14:19.865ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2f285858-1c0a-4a23-8f3a-5d1cb7a38c70", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
38032025-04-30T06:14:19.866ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
38042025-04-30T06:14:19.867ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a2df48db-3187-4647-89c5-fd885ae2ff79", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
38052025-04-30T06:14:19.868ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
38062025-04-30T06:14:19.869ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "488d1d63-b717-4360-9dd7-225d18f51188", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
38072025-04-30T06:14:19.870ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
38082025-04-30T06:14:19.871ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7eeca68c-858b-48a3-a0ae-dceabe44041f", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
38092025-04-30T06:14:19.872ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0
38102025-04-30T06:14:19.875ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2810fc33-bbdc-4730-95be-da5a80dfbddd", "content-length": "723", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
38112025-04-30T06:14:19.875ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=0
38122025-04-30T06:14:19.883ZWARNtest_basic (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
38132025-04-30T06:14:19.886ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ec297a7b-81a2-4852-99a9-68ef95bb6ffe", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
38142025-04-30T06:14:19.886ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=1
38152025-04-30T06:14:19.889ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "34cc9c71-6ac6-429c-95c7-f6afcff13749", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
38162025-04-30T06:14:19.889ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=0
38172025-04-30T06:14:19.893ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "394ad66a-992d-41af-aa77-a24185334853", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
38182025-04-30T06:14:19.893ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=1
38192025-04-30T06:14:19.897ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "91b8ecc4-39f8-4529-8f60-fae92c2dc98c", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
38202025-04-30T06:14:19.897ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=0
38212025-04-30T06:14:19.906ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8f2b0699-6a79-41a9-ab26-40e4d7c9530a", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
38222025-04-30T06:14:19.906ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=1
38232025-04-30T06:14:19.920ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e646ac8a-b932-4000-aa92-59342c86a4ff", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
38242025-04-30T06:14:19.920ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cmpa
38252025-04-30T06:14:19.922ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "895b336d-fafd-415f-ac8f-275bdf8bae5d", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
38262025-04-30T06:14:19.922ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
38272025-04-30T06:14:19.924ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6b19a976-f000-4859-8716-ae4115db4d5c", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
38282025-04-30T06:14:19.924ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
38292025-04-30T06:14:19.927ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7d12aa5e-5c06-4258-989b-6b16f970fe05", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
38302025-04-30T06:14:19.927ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
38312025-04-30T06:14:19.928ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5770baa6-f760-4a46-9da5-9da5779edcb4", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
38322025-04-30T06:14:19.929ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1
38332025-04-30T06:14:19.931ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "62ee2eca-9ea5-48ab-ab51-1aceab00c38c", "content-length": "723", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
38342025-04-30T06:14:19.931ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=0
38352025-04-30T06:14:19.936ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "36ee976d-092b-49b0-8654-59c3b4d718ec", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
38362025-04-30T06:14:19.936ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=1
38372025-04-30T06:14:19.940ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ac25731e-318c-4485-b2e7-d34623dec1f1", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
38382025-04-30T06:14:19.940ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=0
38392025-04-30T06:14:19.944ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0342eabc-261a-41ae-882a-c80c42076343", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
38402025-04-30T06:14:19.944ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=1
38412025-04-30T06:14:19.947ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6e8149ed-4ca9-4aa7-be12-8bb5ac7f4b82", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
38422025-04-30T06:14:19.947ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=0
38432025-04-30T06:14:19.954ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "452a86e4-69c9-4753-abce-6dd64662135b", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
38442025-04-30T06:14:19.954ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=1
38452025-04-30T06:14:19.958ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d12efdf4-04c3-4e28-ab30-037060298623", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
38462025-04-30T06:14:19.958ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cmpa
38472025-04-30T06:14:19.960ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "84a0d8fd-f265-450b-b3d3-2558f17949b9", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
38482025-04-30T06:14:19.960ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
38492025-04-30T06:14:19.961ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8baeffb6-bfec-4c43-9544-696eb32e370a", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
38502025-04-30T06:14:19.961ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
38512025-04-30T06:14:19.963ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a9147608-9d30-4ca4-857c-20802b6413cb", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
38522025-04-30T06:14:19.963ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
38532025-04-30T06:14:19.965ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "43d94001-3505-44a2-b6bf-6f0135f3dbdb", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:19 GMT"} })
38542025-04-30T06:14:19.965ZTRACtest_basic: authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet
38552025-04-30T06:14:19.965ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
38562025-04-30T06:14:19.965ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
38572025-04-30T06:14:19.965ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
38582025-04-30T06:14:19.966ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
38592025-04-30T06:14:19.971ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38602025-04-30T06:14:19.976ZDEBGtest_basic: authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet result = Ok(())
38612025-04-30T06:14:19.976ZTRACtest_basic: authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet
38622025-04-30T06:14:19.976ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
38632025-04-30T06:14:19.976ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
38642025-04-30T06:14:19.976ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
38652025-04-30T06:14:19.977ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
38662025-04-30T06:14:19.984ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38672025-04-30T06:14:19.990ZDEBGtest_basic: authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet result = Ok(())
38682025-04-30T06:14:19.990ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
38692025-04-30T06:14:19.990ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
38702025-04-30T06:14:19.990ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
38712025-04-30T06:14:20.055ZDEBGtest_basic: begin collection from Sled Agent sled_agent_url = http://[::1]:42420
38722025-04-30T06:14:20.055ZDEBGtest_basic: client request SledAgent = http://[::1]:42420 body = None method = GET uri = http://[::1]:42420/inventory
38732025-04-30T06:14:20.056ZINFOtest_basic (dropshot): accepted connection local_addr = [::1]:42420 remote_addr = [::1]:43416 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
38742025-04-30T06:14:20.057ZTRACtest_basic (dropshot): incoming request local_addr = [::1]:42420 method = GET remote_addr = [::1]:43416 req_id = 235b48e2-4dd6-45b2-8ac8-69c066c38905 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
38752025-04-30T06:14:20.058ZINFOtest_basic (dropshot): request completed latency_us = 560 local_addr = [::1]:42420 method = GET remote_addr = [::1]:43416 req_id = 235b48e2-4dd6-45b2-8ac8-69c066c38905 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
38762025-04-30T06:14:20.058ZDEBGtest_basic: client response SledAgent = http://[::1]:42420 result = Ok(Response { url: "http://[::1]:42420/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "235b48e2-4dd6-45b2-8ac8-69c066c38905", "content-length": "2769", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
38772025-04-30T06:14:20.061ZDEBGtest_basic: begin collecting all keepers nkeeper_admin_clients = 0
38782025-04-30T06:14:20.061ZDEBGtest_basic: end collecting all keepers nkeeper_admin_clients = 0
38792025-04-30T06:14:20.061ZDEBGtest_basic: finished collection
38802025-04-30T06:14:20.063ZTRACtest_basic: authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory
38812025-04-30T06:14:20.063ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
38822025-04-30T06:14:20.063ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
38832025-04-30T06:14:20.063ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
38842025-04-30T06:14:20.064ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
38852025-04-30T06:14:20.071ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38862025-04-30T06:14:20.078ZDEBGtest_basic: authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory result = Ok(())
38872025-04-30T06:14:20.078ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
38882025-04-30T06:14:20.078ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
38892025-04-30T06:14:20.079ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
38902025-04-30T06:14:20.158ZWARNtest_basic (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
38912025-04-30T06:14:20.158ZWARNtest_basic: inventory collection failed error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
38922025-04-30T06:14:20.165ZTRACtest_basic: authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory
38932025-04-30T06:14:20.165ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
38942025-04-30T06:14:20.165ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
38952025-04-30T06:14:20.165ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
38962025-04-30T06:14:20.167ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
38972025-04-30T06:14:20.172ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38982025-04-30T06:14:20.178ZDEBGtest_basic: authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory result = Ok(())
38992025-04-30T06:14:20.178ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
39002025-04-30T06:14:20.178ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
39012025-04-30T06:14:20.179ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
39022025-04-30T06:14:20.186ZDEBGtest_basic: inventory_prune_one: nothing eligible for removal (too few) candidates = [(efcb0ee5-cafa-4692-9b75-fcee4732bdca, 0)]
39032025-04-30T06:14:20.187ZTRACtest_basic: lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
39042025-04-30T06:14:20.190ZTRACtest_basic (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 3b f5 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ;............_mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:62086 req_id = 91541e31-869a-4627-9be3-8583590d6138
39052025-04-30T06:14:20.191ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 15349,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62086 req_id = 91541e31-869a-4627-9be3-8583590d6138
39062025-04-30T06:14:20.191ZDEBGtest_basic (store): query key key = _mgs._tcp
39072025-04-30T06:14:20.192ZDEBGtest_basic (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
39082025-04-30T06:14:20.192ZDEBGtest_basic (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:62086 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 37822, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 91541e31-869a-4627-9be3-8583590d6138
39092025-04-30T06:14:20.192ZDEBGtest_basic: lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 37822, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2524, tv_nsec: 991628832 } })
39102025-04-30T06:14:20.192ZTRACtest_basic: lookup_service_targets: looking up SRV target
39112025-04-30T06:14:20.193ZTRACtest_basic (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 44 92 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e D...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:56561 req_id = 1752c596-39ae-4a82-b60c-b3d563f277eb
39122025-04-30T06:14:20.193ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 17554,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:56561 req_id = 1752c596-39ae-4a82-b60c-b3d563f277eb
39132025-04-30T06:14:20.193ZDEBGtest_basic (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
39142025-04-30T06:14:20.193ZDEBGtest_basic (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:56561 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 1752c596-39ae-4a82-b60c-b3d563f277eb
39152025-04-30T06:14:20.275ZTRACtest_basic: lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
39162025-04-30T06:14:20.276ZTRACtest_basic (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: a7 d1 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:63890 req_id = 76816cf1-6fa5-40c0-b8c0-14454f4709ee
39172025-04-30T06:14:20.276ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 42961,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63890 req_id = 76816cf1-6fa5-40c0-b8c0-14454f4709ee
39182025-04-30T06:14:20.276ZDEBGtest_basic (store): query key key = _clickhouse-admin-keeper._tcp
39192025-04-30T06:14:20.277ZERROtest_basic (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 42961, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:63890 req_id = 76816cf1-6fa5-40c0-b8c0-14454f4709ee
39202025-04-30T06:14:20.277ZDEBGtest_basic: begin collection
39212025-04-30T06:14:20.277ZDEBGtest_basic: begin collection from MGS mgs_url = http://[::1]:37822
39222025-04-30T06:14:20.277ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/ignition
39232025-04-30T06:14:20.286ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "78b2af50-81b7-48fa-9d63-e90b3205bc13", "content-length": "835", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
39242025-04-30T06:14:20.286ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0
39252025-04-30T06:14:20.288ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "261b7c03-c2b5-4145-95b1-15dd9e92f611", "content-length": "734", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
39262025-04-30T06:14:20.288ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=0
39272025-04-30T06:14:20.291ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "18e225d2-add9-4e8e-9759-50a0edcbe1a9", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
39282025-04-30T06:14:20.291ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=1
39292025-04-30T06:14:20.300ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ed13a742-6238-47e0-bd66-3bd075602834", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
39302025-04-30T06:14:20.300ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=0
39312025-04-30T06:14:20.304ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4f1dee38-bbfe-4d8f-ad11-ea8b8eded9e4", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
39322025-04-30T06:14:20.305ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=1
39332025-04-30T06:14:20.313ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d22317cf-5d27-4a6a-9568-41f41bd562ea", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
39342025-04-30T06:14:20.313ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=0
39352025-04-30T06:14:20.317ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5ce68513-66d3-47d3-91f9-94c20e636ce5", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
39362025-04-30T06:14:20.317ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=1
39372025-04-30T06:14:20.319ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bfab3d22-16d5-489d-9ef4-0661764422fb", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
39382025-04-30T06:14:20.320ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cmpa
39392025-04-30T06:14:20.321ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "08b1e5d5-0dab-4439-a2b6-9ee1da2d4a95", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
39402025-04-30T06:14:20.321ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
39412025-04-30T06:14:20.323ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e999c30c-cde6-4fb7-9890-6e0b7c7bd03a", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
39422025-04-30T06:14:20.323ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
39432025-04-30T06:14:20.324ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b4a7d5fa-2843-4e3d-882b-493f87cdce8e", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
39442025-04-30T06:14:20.325ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
39452025-04-30T06:14:20.327ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "98952241-7568-411d-9403-266e5ba38ab3", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
39462025-04-30T06:14:20.327ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1
39472025-04-30T06:14:20.331ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "eb98a19c-11c4-4f17-87de-1d9367eaf36d", "content-length": "734", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
39482025-04-30T06:14:20.331ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=0
39492025-04-30T06:14:20.335ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "eaa9f27f-7277-498e-a952-5d780945adf6", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
39502025-04-30T06:14:20.335ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=1
39512025-04-30T06:14:20.337ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f2ffb47a-c2c2-4386-86d8-3e93f9f77585", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
39522025-04-30T06:14:20.337ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=0
39532025-04-30T06:14:20.340ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6438b00c-f375-46d4-bd4a-8aca100bc926", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
39542025-04-30T06:14:20.340ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=1
39552025-04-30T06:14:20.342ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2419f89c-94f2-4e20-8770-b0e2f0bec2c3", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
39562025-04-30T06:14:20.342ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=0
39572025-04-30T06:14:20.344ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a5f5228b-741c-454e-99e7-6a420234c640", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
39582025-04-30T06:14:20.345ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=1
39592025-04-30T06:14:20.349ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d926236f-cc59-4d4f-96f8-e114462dc4a1", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
39602025-04-30T06:14:20.349ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cmpa
39612025-04-30T06:14:20.350ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6de72991-3a10-4617-a3b3-b0f35ae60bc6", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
39622025-04-30T06:14:20.351ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
39632025-04-30T06:14:20.353ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cce1e38f-66c7-452d-a169-01c581d79e16", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
39642025-04-30T06:14:20.353ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
39652025-04-30T06:14:20.355ZDEBGtest_basic (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
39662025-04-30T06:14:20.355ZDEBGtest_basic (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
39672025-04-30T06:14:20.359ZTRACtest_basic (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
39682025-04-30T06:14:20.361ZDEBGtest_basic (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
39692025-04-30T06:14:20.361ZTRACtest_basic (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
39702025-04-30T06:14:20.361ZTRACtest_basic (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
39712025-04-30T06:14:20.363ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ba41bee4-ff98-4ad4-a4b9-3226fe5b0d58", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
39722025-04-30T06:14:20.364ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
39732025-04-30T06:14:20.366ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ad4c15a9-9b84-42c4-b138-bb2b594e18bf", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
39742025-04-30T06:14:20.367ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0
39752025-04-30T06:14:20.371ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1fffbcc0-5557-42c2-a7d3-8da96519cfe7", "content-length": "723", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
39762025-04-30T06:14:20.372ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=0
39772025-04-30T06:14:20.378ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9365c050-b41c-41c2-8ccf-26515edef931", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
39782025-04-30T06:14:20.378ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=1
39792025-04-30T06:14:20.382ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "08900d66-5400-4f49-9f48-f57f5aaa740e", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
39802025-04-30T06:14:20.382ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=0
39812025-04-30T06:14:20.386ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "75e7e940-1eae-4014-8726-7d02441f2fc6", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
39822025-04-30T06:14:20.386ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=1
39832025-04-30T06:14:20.391ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2b192b3b-ef55-4eb2-b184-8485ec3636a2", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
39842025-04-30T06:14:20.391ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=0
39852025-04-30T06:14:20.396ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4a651a24-085a-42b2-9675-1330c6753699", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
39862025-04-30T06:14:20.397ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=1
39872025-04-30T06:14:20.405ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b329cc6c-a98c-4236-819b-df4d1d5fcdc8", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
39882025-04-30T06:14:20.406ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cmpa
39892025-04-30T06:14:20.413ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a014bc23-b0de-48a7-a9e5-a8cea51062d2", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
39902025-04-30T06:14:20.414ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
39912025-04-30T06:14:20.417ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0a6b5d74-c872-44a5-adba-984665d471ab", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
39922025-04-30T06:14:20.418ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
39932025-04-30T06:14:20.426ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "be74a6e8-c20f-4486-a420-49f2111a82d3", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
39942025-04-30T06:14:20.427ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
39952025-04-30T06:14:20.430ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4b56460c-8ea7-4909-9b3a-fc19f4e1b1f2", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
39962025-04-30T06:14:20.440ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1
39972025-04-30T06:14:20.441ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "12059380-271a-4b20-a39f-14bdc79a8fc2", "content-length": "723", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
39982025-04-30T06:14:20.441ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=0
39992025-04-30T06:14:20.449ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "40e224d5-a3f5-429b-a510-420c95ef95ab", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
40002025-04-30T06:14:20.449ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=1
40012025-04-30T06:14:20.453ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "03a74edc-f6bd-4f39-8ee4-e7d3456c8be2", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
40022025-04-30T06:14:20.453ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=0
40032025-04-30T06:14:20.456ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f0277393-cc4b-4c8e-a699-189802e1e661", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
40042025-04-30T06:14:20.456ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=1
40052025-04-30T06:14:20.459ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fae9c432-6a76-49af-9024-d7a44dbac8d1", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
40062025-04-30T06:14:20.460ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=0
40072025-04-30T06:14:20.469ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a9a7df41-b2a5-49ed-9a98-ba2698d98b2a", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
40082025-04-30T06:14:20.471ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=1
40092025-04-30T06:14:20.477ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1e8fa603-913a-4221-b85d-6eb2d4be2ca1", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
40102025-04-30T06:14:20.477ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cmpa
40112025-04-30T06:14:20.479ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3d482794-6341-44a6-96b1-d75721e7f5c8", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
40122025-04-30T06:14:20.479ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
40132025-04-30T06:14:20.482ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3b484f7d-02ef-4e81-a513-b884cd3fd3cb", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
40142025-04-30T06:14:20.482ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
40152025-04-30T06:14:20.484ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0c011b8-c436-46b8-b7d0-39e270ea0ca9", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
40162025-04-30T06:14:20.485ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
40172025-04-30T06:14:20.486ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "86764018-0bef-41a3-86b7-94df870374a4", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
40182025-04-30T06:14:20.486ZTRACtest_basic: authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet
40192025-04-30T06:14:20.486ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
40202025-04-30T06:14:20.486ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
40212025-04-30T06:14:20.488ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
40222025-04-30T06:14:20.488ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
40232025-04-30T06:14:20.493ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40242025-04-30T06:14:20.504ZDEBGtest_basic: authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet result = Ok(())
40252025-04-30T06:14:20.504ZTRACtest_basic: authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet
40262025-04-30T06:14:20.504ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
40272025-04-30T06:14:20.504ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
40282025-04-30T06:14:20.504ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
40292025-04-30T06:14:20.506ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
40302025-04-30T06:14:20.513ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40312025-04-30T06:14:20.522ZDEBGtest_basic: authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet result = Ok(())
40322025-04-30T06:14:20.523ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
40332025-04-30T06:14:20.523ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
40342025-04-30T06:14:20.523ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
40352025-04-30T06:14:20.580ZDEBGtest_basic: begin collection from Sled Agent sled_agent_url = http://[::1]:42420
40362025-04-30T06:14:20.580ZDEBGtest_basic: client request SledAgent = http://[::1]:42420 body = None method = GET uri = http://[::1]:42420/inventory
40372025-04-30T06:14:20.581ZINFOtest_basic (dropshot): accepted connection local_addr = [::1]:42420 remote_addr = [::1]:48018 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
40382025-04-30T06:14:20.582ZTRACtest_basic (dropshot): incoming request local_addr = [::1]:42420 method = GET remote_addr = [::1]:48018 req_id = e5f0742a-ad03-43a9-9c07-8b08513e6748 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
40392025-04-30T06:14:20.582ZINFOtest_basic (dropshot): request completed latency_us = 474 local_addr = [::1]:42420 method = GET remote_addr = [::1]:48018 req_id = e5f0742a-ad03-43a9-9c07-8b08513e6748 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
40402025-04-30T06:14:20.583ZDEBGtest_basic: client response SledAgent = http://[::1]:42420 result = Ok(Response { url: "http://[::1]:42420/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "e5f0742a-ad03-43a9-9c07-8b08513e6748", "content-length": "2769", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
40412025-04-30T06:14:20.584ZDEBGtest_basic: begin collecting all keepers nkeeper_admin_clients = 0
40422025-04-30T06:14:20.584ZDEBGtest_basic: end collecting all keepers nkeeper_admin_clients = 0
40432025-04-30T06:14:20.584ZDEBGtest_basic: finished collection
40442025-04-30T06:14:20.589ZTRACtest_basic: authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory
40452025-04-30T06:14:20.589ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
40462025-04-30T06:14:20.589ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
40472025-04-30T06:14:20.589ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
40482025-04-30T06:14:20.590ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
40492025-04-30T06:14:20.594ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40502025-04-30T06:14:20.599ZDEBGtest_basic: authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory result = Ok(())
40512025-04-30T06:14:20.599ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
40522025-04-30T06:14:20.599ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
40532025-04-30T06:14:20.600ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
40542025-04-30T06:14:20.628ZWARNtest_basic (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
40552025-04-30T06:14:20.629ZWARNtest_basic: inventory collection failed error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
40562025-04-30T06:14:20.636ZTRACtest_basic: authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory
40572025-04-30T06:14:20.636ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
40582025-04-30T06:14:20.636ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
40592025-04-30T06:14:20.636ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
40602025-04-30T06:14:20.637ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
40612025-04-30T06:14:20.641ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40622025-04-30T06:14:20.649ZDEBGtest_basic: authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory result = Ok(())
40632025-04-30T06:14:20.649ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
40642025-04-30T06:14:20.649ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
40652025-04-30T06:14:20.650ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
40662025-04-30T06:14:20.653ZDEBGtest_basic: inventory_prune_one: nothing eligible for removal (too few) candidates = [(efcb0ee5-cafa-4692-9b75-fcee4732bdca, 0)]
40672025-04-30T06:14:20.653ZTRACtest_basic: lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
40682025-04-30T06:14:20.656ZTRACtest_basic (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 5a ab 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 Z............_mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:54772 req_id = faaeb957-de06-43bf-a71c-be753a01fb9e
40692025-04-30T06:14:20.667ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 23211,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54772 req_id = faaeb957-de06-43bf-a71c-be753a01fb9e
40702025-04-30T06:14:20.669ZDEBGtest_basic (store): query key key = _mgs._tcp
40712025-04-30T06:14:20.670ZDEBGtest_basic (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
40722025-04-30T06:14:20.670ZDEBGtest_basic (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:54772 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 37822, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = faaeb957-de06-43bf-a71c-be753a01fb9e
40732025-04-30T06:14:20.671ZDEBGtest_basic: lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 37822, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2525, tv_nsec: 470549911 } })
40742025-04-30T06:14:20.671ZTRACtest_basic: lookup_service_targets: looking up SRV target
40752025-04-30T06:14:20.672ZTRACtest_basic (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 04 9c 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:60689 req_id = 3d705733-c714-47b5-998b-faab142e500d
40762025-04-30T06:14:20.672ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 1180,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60689 req_id = 3d705733-c714-47b5-998b-faab142e500d
40772025-04-30T06:14:20.672ZDEBGtest_basic (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
40782025-04-30T06:14:20.672ZDEBGtest_basic (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:60689 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 3d705733-c714-47b5-998b-faab142e500d
40792025-04-30T06:14:20.755ZTRACtest_basic: lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
40802025-04-30T06:14:20.755ZDEBGtest_basic (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
40812025-04-30T06:14:20.755ZDEBGtest_basic (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
40822025-04-30T06:14:20.756ZDEBGtest_basic (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812
40832025-04-30T06:14:20.757ZTRACtest_basic (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
40842025-04-30T06:14:20.759ZTRACtest_basic (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 15 a2 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:51732 req_id = 2794250a-4c70-4a8a-b019-f8b24a082eab
40852025-04-30T06:14:20.759ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 5538,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51732 req_id = 2794250a-4c70-4a8a-b019-f8b24a082eab
40862025-04-30T06:14:20.759ZDEBGtest_basic (store): query key key = _clickhouse-admin-keeper._tcp
40872025-04-30T06:14:20.759ZERROtest_basic (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 5538, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:51732 req_id = 2794250a-4c70-4a8a-b019-f8b24a082eab
40882025-04-30T06:14:20.759ZDEBGtest_basic: begin collection
40892025-04-30T06:14:20.759ZDEBGtest_basic: begin collection from MGS mgs_url = http://[::1]:37822
40902025-04-30T06:14:20.760ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/ignition
40912025-04-30T06:14:20.764ZDEBGtest_basic (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_rows = 2 table_name = fields_string
40922025-04-30T06:14:20.764ZTRACtest_basic (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
40932025-04-30T06:14:20.766ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "a48b9f3d-f929-4639-9c94-ea8798ec9d77", "content-length": "835", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
40942025-04-30T06:14:20.766ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0
40952025-04-30T06:14:20.771ZDEBGtest_basic (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_rows = 1 table_name = measurements_i64
40962025-04-30T06:14:20.771ZTRACtest_basic (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
40972025-04-30T06:14:20.771ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ab91d56e-298e-42e0-abc2-e90aea776f7c", "content-length": "734", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
40982025-04-30T06:14:20.772ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=0
40992025-04-30T06:14:20.774ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d4959869-93d2-4a12-aedf-626fbf858b3f", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
41002025-04-30T06:14:20.774ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=1
41012025-04-30T06:14:20.776ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5257afba-249a-4d17-aada-3f5865162217", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
41022025-04-30T06:14:20.777ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=0
41032025-04-30T06:14:20.779ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "edb5fe9f-3da5-47a8-b3f7-44ad589e199f", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
41042025-04-30T06:14:20.779ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=1
41052025-04-30T06:14:20.782ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "90882f54-16c4-4235-bbfc-5f0741f543cd", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
41062025-04-30T06:14:20.782ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=0
41072025-04-30T06:14:20.784ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "91ee7d04-65d5-4dda-a112-872cd65b76c3", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
41082025-04-30T06:14:20.784ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=1
41092025-04-30T06:14:20.786ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f1373e9c-7b0b-4975-8905-f5fcef7f8283", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
41102025-04-30T06:14:20.786ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cmpa
41112025-04-30T06:14:20.787ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "acc404f1-99b8-45b7-82e9-d1a1e209f2a8", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
41122025-04-30T06:14:20.788ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
41132025-04-30T06:14:20.790ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1c207c38-9085-41a8-8d86-d5f381c5661b", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
41142025-04-30T06:14:20.790ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
41152025-04-30T06:14:20.793ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4436624c-bb9a-48c3-802d-6019e25843a7", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
41162025-04-30T06:14:20.793ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
41172025-04-30T06:14:20.794ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d17375d1-a04e-48c2-8673-0fc75d652a5b", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
41182025-04-30T06:14:20.794ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1
41192025-04-30T06:14:20.796ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "47e6b8f1-c08c-4dab-a65e-f61e54a1d2e0", "content-length": "734", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
41202025-04-30T06:14:20.797ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=0
41212025-04-30T06:14:20.799ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "70d80fc3-9176-4e00-b966-7df06e7685d3", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
41222025-04-30T06:14:20.799ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=1
41232025-04-30T06:14:20.801ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c9c1ae54-adc5-4fcf-9461-e2bafe2890c2", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
41242025-04-30T06:14:20.801ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=0
41252025-04-30T06:14:20.804ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0b965176-d769-4773-b308-7aea8ab4c7bb", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
41262025-04-30T06:14:20.804ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=1
41272025-04-30T06:14:20.808ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6673fa89-9b77-4c63-88b0-1b5768035464", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
41282025-04-30T06:14:20.808ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=0
41292025-04-30T06:14:20.811ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "aaf3972f-0bec-43d8-9cd2-b15afbd3b030", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
41302025-04-30T06:14:20.811ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=1
41312025-04-30T06:14:20.814ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ea0c2244-ca3b-437b-b43a-c619d3d409a3", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
41322025-04-30T06:14:20.814ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cmpa
41332025-04-30T06:14:20.815ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c2ff790b-4061-44a7-9e33-fb0d05bff6c8", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
41342025-04-30T06:14:20.815ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
41352025-04-30T06:14:20.817ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "200b5444-6418-47f0-858a-f05773f6cd8a", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
41362025-04-30T06:14:20.817ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
41372025-04-30T06:14:20.820ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0d818ef3-cbc3-439e-a534-e8755fff86d3", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
41382025-04-30T06:14:20.820ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
41392025-04-30T06:14:20.823ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7e75c77f-e4f0-4cb1-8111-ff77cc6a25dd", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
41402025-04-30T06:14:20.823ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0
41412025-04-30T06:14:20.827ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f137edd6-d7b7-4731-8787-ab66ac5f484f", "content-length": "723", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
41422025-04-30T06:14:20.827ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=0
41432025-04-30T06:14:20.832ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "03c8ea49-8f24-46c2-b68a-18b408ed22ca", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
41442025-04-30T06:14:20.832ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=1
41452025-04-30T06:14:20.837ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e73ec260-8712-4e47-9dcb-afba8d81cef6", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
41462025-04-30T06:14:20.837ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=0
41472025-04-30T06:14:20.841ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "36f3960c-2e3e-4dfe-b822-4b6bd5011bf0", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
41482025-04-30T06:14:20.841ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=1
41492025-04-30T06:14:20.847ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "af25695e-73f4-4520-bf17-fe2418046ea1", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
41502025-04-30T06:14:20.847ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=0
41512025-04-30T06:14:20.852ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0dd9a397-bf5f-4032-b072-f8f9434e8e37", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
41522025-04-30T06:14:20.852ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=1
41532025-04-30T06:14:20.856ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "664b3883-a163-419b-935f-62fe0922a0a4", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
41542025-04-30T06:14:20.856ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cmpa
41552025-04-30T06:14:20.857ZWARNtest_basic (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
41562025-04-30T06:14:20.858ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fd3e8642-1188-4653-b4e0-5d712be5c131", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
41572025-04-30T06:14:20.859ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
41582025-04-30T06:14:20.861ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc66723d-1983-4de1-b2f1-ff70b69ff846", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
41592025-04-30T06:14:20.861ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
41602025-04-30T06:14:20.866ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7e6920d9-1ca2-4f96-940c-0844cf71a950", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
41612025-04-30T06:14:20.866ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
41622025-04-30T06:14:20.868ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c6944405-c21b-441c-b5cd-2dd0f862822c", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
41632025-04-30T06:14:20.868ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1
41642025-04-30T06:14:20.869ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "06745a82-2a65-4196-a880-9202ccf861dc", "content-length": "723", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
41652025-04-30T06:14:20.870ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=0
41662025-04-30T06:14:20.874ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b75cb894-6a38-4904-b517-7fa1a76f6297", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
41672025-04-30T06:14:20.874ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=1
41682025-04-30T06:14:20.877ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ab904f7e-6a30-437c-841c-0a38d549089b", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
41692025-04-30T06:14:20.877ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=0
41702025-04-30T06:14:20.880ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f62385dd-147a-4c77-9c31-d9c816bc06fb", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
41712025-04-30T06:14:20.880ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=1
41722025-04-30T06:14:20.883ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7184e103-9d75-4cee-8ea2-4057fa566a3c", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
41732025-04-30T06:14:20.883ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=0
41742025-04-30T06:14:20.891ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "052e5f3b-3ed7-4134-bacc-7946881330fc", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
41752025-04-30T06:14:20.891ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=1
41762025-04-30T06:14:20.894ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "23a45a4e-1e0a-476d-8cdc-77223b6d85e5", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
41772025-04-30T06:14:20.894ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cmpa
41782025-04-30T06:14:20.896ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "77865d4c-5fd6-4a29-bbac-b3a2b90b4439", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
41792025-04-30T06:14:20.896ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
41802025-04-30T06:14:20.897ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "040e278e-fc13-40f3-a7ab-153257e1751d", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
41812025-04-30T06:14:20.898ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
41822025-04-30T06:14:20.899ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9de6e65f-2be6-4b21-b2b0-72a94bcb3909", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
41832025-04-30T06:14:20.900ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
41842025-04-30T06:14:20.901ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6e1f77ef-279b-4374-a714-81bb82f4c9ac", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
41852025-04-30T06:14:20.901ZTRACtest_basic: authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet
41862025-04-30T06:14:20.901ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
41872025-04-30T06:14:20.901ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
41882025-04-30T06:14:20.901ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
41892025-04-30T06:14:20.903ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
41902025-04-30T06:14:20.908ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41912025-04-30T06:14:20.916ZDEBGtest_basic: authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet result = Ok(())
41922025-04-30T06:14:20.916ZTRACtest_basic: authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet
41932025-04-30T06:14:20.916ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
41942025-04-30T06:14:20.916ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
41952025-04-30T06:14:20.916ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
41962025-04-30T06:14:20.917ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
41972025-04-30T06:14:20.924ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41982025-04-30T06:14:20.933ZDEBGtest_basic: authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet result = Ok(())
41992025-04-30T06:14:20.934ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
42002025-04-30T06:14:20.934ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
42012025-04-30T06:14:20.934ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
42022025-04-30T06:14:21.033ZDEBGtest_basic: begin collection from Sled Agent sled_agent_url = http://[::1]:42420
42032025-04-30T06:14:21.033ZDEBGtest_basic: client request SledAgent = http://[::1]:42420 body = None method = GET uri = http://[::1]:42420/inventory
42042025-04-30T06:14:21.035ZINFOtest_basic (dropshot): accepted connection local_addr = [::1]:42420 remote_addr = [::1]:63296 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
42052025-04-30T06:14:21.035ZINFOtest_basic (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
42062025-04-30T06:14:21.037ZDEBGtest_basic (oximeter): client request body = None method = GET uri = http://127.0.0.1:34211/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
42072025-04-30T06:14:21.038ZTRACtest_basic (dropshot_internal): incoming request local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 457fe7b0-af00-4b81-aac1-371de3e02c9c uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
42082025-04-30T06:14:21.040ZTRACtest_basic (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 457fe7b0-af00-4b81-aac1-371de3e02c9c resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
42092025-04-30T06:14:21.040ZDEBGtest_basic (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 457fe7b0-af00-4b81-aac1-371de3e02c9c roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
42102025-04-30T06:14:21.041ZDEBGtest_basic (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 457fe7b0-af00-4b81-aac1-371de3e02c9c resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
42112025-04-30T06:14:21.042ZTRACtest_basic (dropshot): incoming request local_addr = [::1]:42420 method = GET remote_addr = [::1]:63296 req_id = a6a50c31-5405-42d4-8676-ab651f7c2c29 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
42122025-04-30T06:14:21.043ZINFOtest_basic (dropshot): request completed latency_us = 914 local_addr = [::1]:42420 method = GET remote_addr = [::1]:63296 req_id = a6a50c31-5405-42d4-8676-ab651f7c2c29 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
42132025-04-30T06:14:21.046ZDEBGtest_basic: client response SledAgent = http://[::1]:42420 result = Ok(Response { url: "http://[::1]:42420/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "a6a50c31-5405-42d4-8676-ab651f7c2c29", "content-length": "2769", "date": "Wed, 30 Apr 2025 06:14:20 GMT"} })
42142025-04-30T06:14:21.051ZDEBGtest_basic: begin collecting all keepers nkeeper_admin_clients = 0
42152025-04-30T06:14:21.051ZDEBGtest_basic: end collecting all keepers nkeeper_admin_clients = 0
42162025-04-30T06:14:21.051ZDEBGtest_basic: finished collection
42172025-04-30T06:14:21.053ZTRACtest_basic: authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory
42182025-04-30T06:14:21.053ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
42192025-04-30T06:14:21.054ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
42202025-04-30T06:14:21.054ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
42212025-04-30T06:14:21.055ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
42222025-04-30T06:14:21.057ZINFOtest_basic (dropshot_internal): request completed latency_us = 19055 local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 457fe7b0-af00-4b81-aac1-371de3e02c9c response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
42232025-04-30T06:14:21.067ZDEBGtest_basic (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:34211/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "457fe7b0-af00-4b81-aac1-371de3e02c9c", "content-length": "405", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
42242025-04-30T06:14:21.068ZDEBGtest_basic (oximeter): client request body = None method = GET uri = http://127.0.0.1:34211/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
42252025-04-30T06:14:21.068ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42262025-04-30T06:14:21.079ZDEBGtest_basic: authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory result = Ok(())
42272025-04-30T06:14:21.080ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
42282025-04-30T06:14:21.080ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
42292025-04-30T06:14:21.082ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
42302025-04-30T06:14:21.082ZTRACtest_basic (dropshot_internal): incoming request local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 89e6eeea-ab2e-4e78-8dd7-979f0d9d4a9c uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
42312025-04-30T06:14:21.083ZTRACtest_basic (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 89e6eeea-ab2e-4e78-8dd7-979f0d9d4a9c resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
42322025-04-30T06:14:21.083ZDEBGtest_basic (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 89e6eeea-ab2e-4e78-8dd7-979f0d9d4a9c roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
42332025-04-30T06:14:21.085ZDEBGtest_basic (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 89e6eeea-ab2e-4e78-8dd7-979f0d9d4a9c resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
42342025-04-30T06:14:21.090ZINFOtest_basic (dropshot_internal): request completed latency_us = 8454 local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 89e6eeea-ab2e-4e78-8dd7-979f0d9d4a9c response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
42352025-04-30T06:14:21.092ZDEBGtest_basic (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:34211/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "89e6eeea-ab2e-4e78-8dd7-979f0d9d4a9c", "content-length": "29", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
42362025-04-30T06:14:21.092ZTRACtest_basic (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 088ac96c-981b-4054-bea6-d8ab23ae1366
42372025-04-30T06:14:21.092ZTRACtest_basic (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
42382025-04-30T06:14:21.092ZINFOtest_basic (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
42392025-04-30T06:14:21.232ZWARNtest_basic (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
42402025-04-30T06:14:21.233ZWARNtest_basic: inventory collection failed error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
42412025-04-30T06:14:21.241ZTRACtest_basic: authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory
42422025-04-30T06:14:21.241ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
42432025-04-30T06:14:21.241ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
42442025-04-30T06:14:21.241ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
42452025-04-30T06:14:21.242ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
42462025-04-30T06:14:21.247ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42472025-04-30T06:14:21.252ZDEBGtest_basic: authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory result = Ok(())
42482025-04-30T06:14:21.252ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
42492025-04-30T06:14:21.252ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
42502025-04-30T06:14:21.253ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
42512025-04-30T06:14:21.269ZDEBGtest_basic: inventory_prune_one: nothing eligible for removal (too few) candidates = [(efcb0ee5-cafa-4692-9b75-fcee4732bdca, 0)]
42522025-04-30T06:14:21.269ZTRACtest_basic: lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
42532025-04-30T06:14:21.271ZTRACtest_basic (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: b6 fc 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:58145 req_id = 3b0e54a8-f883-4288-b969-728a5f32f5b8
42542025-04-30T06:14:21.271ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 46844,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:58145 req_id = 3b0e54a8-f883-4288-b969-728a5f32f5b8
42552025-04-30T06:14:21.271ZDEBGtest_basic (store): query key key = _mgs._tcp
42562025-04-30T06:14:21.271ZDEBGtest_basic (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
42572025-04-30T06:14:21.271ZDEBGtest_basic (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:58145 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 37822, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 3b0e54a8-f883-4288-b969-728a5f32f5b8
42582025-04-30T06:14:21.272ZDEBGtest_basic: lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 37822, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2526, tv_nsec: 71329064 } })
42592025-04-30T06:14:21.272ZTRACtest_basic: lookup_service_targets: looking up SRV target
42602025-04-30T06:14:21.272ZTRACtest_basic (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: f0 0e 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:65235 req_id = bf7e81c9-f48f-4302-b5db-7fd5a35ab354
42612025-04-30T06:14:21.272ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 61454,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:65235 req_id = bf7e81c9-f48f-4302-b5db-7fd5a35ab354
42622025-04-30T06:14:21.272ZDEBGtest_basic (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
42632025-04-30T06:14:21.272ZDEBGtest_basic (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:65235 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = bf7e81c9-f48f-4302-b5db-7fd5a35ab354
42642025-04-30T06:14:21.383ZTRACtest_basic: lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
42652025-04-30T06:14:21.384ZDEBGtest_basic (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
42662025-04-30T06:14:21.384ZDEBGtest_basic (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
42672025-04-30T06:14:21.386ZTRACtest_basic (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 93 39 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c .9..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:64702 req_id = 65899a79-c711-4361-beef-dfcf1a823aa6
42682025-04-30T06:14:21.386ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 37689,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64702 req_id = 65899a79-c711-4361-beef-dfcf1a823aa6
42692025-04-30T06:14:21.386ZDEBGtest_basic (store): query key key = _clickhouse-admin-keeper._tcp
42702025-04-30T06:14:21.386ZERROtest_basic (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 37689, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:64702 req_id = 65899a79-c711-4361-beef-dfcf1a823aa6
42712025-04-30T06:14:21.386ZTRACtest_basic (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
42722025-04-30T06:14:21.387ZDEBGtest_basic (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
42732025-04-30T06:14:21.387ZTRACtest_basic (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
42742025-04-30T06:14:21.387ZTRACtest_basic (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
42752025-04-30T06:14:21.388ZDEBGtest_basic: begin collection
42762025-04-30T06:14:21.388ZDEBGtest_basic: begin collection from MGS mgs_url = http://[::1]:37822
42772025-04-30T06:14:21.388ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/ignition
42782025-04-30T06:14:21.397ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "ace2735f-7e4c-46b9-ac91-dab1cdf432cf", "content-length": "835", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
42792025-04-30T06:14:21.398ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0
42802025-04-30T06:14:21.399ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "28dff4c3-897b-4ed5-bc07-0f74ca6497da", "content-length": "734", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
42812025-04-30T06:14:21.400ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=0
42822025-04-30T06:14:21.402ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fecae5c4-de04-41ca-a75a-0656aa6bcf7d", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
42832025-04-30T06:14:21.402ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=1
42842025-04-30T06:14:21.404ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a867f68-ef3b-4519-b1bb-1ff93cfd8afb", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
42852025-04-30T06:14:21.404ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=0
42862025-04-30T06:14:21.406ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "040d34ec-37cd-452d-9c84-44c07a92ec49", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
42872025-04-30T06:14:21.407ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=1
42882025-04-30T06:14:21.408ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "91586e02-61ec-43ba-aa5d-d9212ac80d64", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
42892025-04-30T06:14:21.409ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=0
42902025-04-30T06:14:21.411ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7e5ac6b8-6bc1-40d7-9763-33c3420f76e6", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
42912025-04-30T06:14:21.411ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=1
42922025-04-30T06:14:21.418ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ba125b12-1a4d-4890-831c-229f3fa2b750", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
42932025-04-30T06:14:21.421ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cmpa
42942025-04-30T06:14:21.422ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9b2d92d6-470b-466a-9ea0-1989f825b243", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
42952025-04-30T06:14:21.423ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
42962025-04-30T06:14:21.425ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8968672c-f275-484b-9513-3cb1df216c6d", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
42972025-04-30T06:14:21.425ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
42982025-04-30T06:14:21.428ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c7d064e4-c4ee-4577-a12a-88fc2bedb8e7", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
42992025-04-30T06:14:21.428ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
43002025-04-30T06:14:21.430ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2e928035-6e0e-472e-bc90-8ab55c0d67eb", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
43012025-04-30T06:14:21.431ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1
43022025-04-30T06:14:21.432ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "db289ba5-a30f-4b32-9802-b0ddc4c246fa", "content-length": "734", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
43032025-04-30T06:14:21.432ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=0
43042025-04-30T06:14:21.437ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "70b8a436-0b75-4478-9a65-7e9034964937", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
43052025-04-30T06:14:21.437ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=1
43062025-04-30T06:14:21.439ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3045df95-02ab-4127-a4d4-54f03f7bf951", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
43072025-04-30T06:14:21.439ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=0
43082025-04-30T06:14:21.441ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e0a6ea76-5c8f-4923-a040-49adf469a9f5", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
43092025-04-30T06:14:21.441ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=1
43102025-04-30T06:14:21.445ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "de70cd98-d338-424a-a670-55b15131fc73", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
43112025-04-30T06:14:21.445ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=0
43122025-04-30T06:14:21.447ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "746472fb-1636-4fdf-8e67-7a4c6b63b64f", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
43132025-04-30T06:14:21.448ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=1
43142025-04-30T06:14:21.449ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "17329598-0973-43c5-b64b-3b8d8918cc0d", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
43152025-04-30T06:14:21.450ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cmpa
43162025-04-30T06:14:21.451ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7c92c80c-e6e4-4ff8-b81d-b3a78d0094d9", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
43172025-04-30T06:14:21.451ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
43182025-04-30T06:14:21.453ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "41b488af-e134-42c8-a551-49fa46d29957", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
43192025-04-30T06:14:21.453ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
43202025-04-30T06:14:21.454ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0dfff054-db00-4be9-aa98-65d099373ff2", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
43212025-04-30T06:14:21.454ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
43222025-04-30T06:14:21.458ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ea46a041-7ab6-4d49-b841-ba874b3a41a8", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
43232025-04-30T06:14:21.458ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0
43242025-04-30T06:14:21.467ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3a428150-3f22-4271-9ce1-7e211d66550a", "content-length": "723", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
43252025-04-30T06:14:21.467ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=0
43262025-04-30T06:14:21.471ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bd37f8cd-88da-46af-916b-8328225d3860", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
43272025-04-30T06:14:21.471ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=1
43282025-04-30T06:14:21.476ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c7469e9c-b1b2-47dd-ad90-4fbbd2361025", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
43292025-04-30T06:14:21.476ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=0
43302025-04-30T06:14:21.481ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e34cca88-f741-49b9-aa6f-bb8bd46b0e50", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
43312025-04-30T06:14:21.481ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=1
43322025-04-30T06:14:21.484ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "24749046-25b3-4e29-8352-f25375b14098", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
43332025-04-30T06:14:21.485ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=0
43342025-04-30T06:14:21.488ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "64da5b4e-753a-4b30-b386-c069c48fe373", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
43352025-04-30T06:14:21.488ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=1
43362025-04-30T06:14:21.491ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "82439671-0337-48a4-ad0f-c3b7c27aad8f", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
43372025-04-30T06:14:21.491ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cmpa
43382025-04-30T06:14:21.492ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "dc8ce21b-5f99-4f17-901b-2c75928d3716", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
43392025-04-30T06:14:21.492ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
43402025-04-30T06:14:21.493ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c376a0c9-de7d-496b-9ff2-05576d037ea6", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
43412025-04-30T06:14:21.494ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
43422025-04-30T06:14:21.495ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1209c832-cbe5-4dfe-b832-49724b1f04e7", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
43432025-04-30T06:14:21.495ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
43442025-04-30T06:14:21.496ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "91f77038-685b-42f7-be81-6270ff432298", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
43452025-04-30T06:14:21.496ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1
43462025-04-30T06:14:21.498ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f506d76d-4acf-49fc-9fa6-b58bf8a1143a", "content-length": "723", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
43472025-04-30T06:14:21.498ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=0
43482025-04-30T06:14:21.501ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fd96619e-1252-40bc-91f9-433f8dc327d6", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
43492025-04-30T06:14:21.501ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=1
43502025-04-30T06:14:21.503ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7983bf84-fe44-4447-91e1-e7ff2d29368d", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
43512025-04-30T06:14:21.504ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=0
43522025-04-30T06:14:21.508ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2951fd55-81a3-413d-bf2b-0057658863d3", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
43532025-04-30T06:14:21.509ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=1
43542025-04-30T06:14:21.511ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "11e494fe-da27-4db7-89d9-c42367abe4ec", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
43552025-04-30T06:14:21.512ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=0
43562025-04-30T06:14:21.520ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "001d5174-d966-42d7-bfd0-5af36786d24b", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
43572025-04-30T06:14:21.520ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=1
43582025-04-30T06:14:21.531ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d4ed6eaa-68f0-4f09-85f4-16fd307bb442", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
43592025-04-30T06:14:21.531ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cmpa
43602025-04-30T06:14:21.534ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b3474599-93c0-4f17-a79d-f4f17ec47056", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
43612025-04-30T06:14:21.534ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
43622025-04-30T06:14:21.536ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f9c9eba4-f6c5-4a8f-934f-62c40c4b7c53", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
43632025-04-30T06:14:21.536ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
43642025-04-30T06:14:21.538ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a37c5c5d-a6b7-40fb-83bf-c8b8a932c058", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
43652025-04-30T06:14:21.538ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
43662025-04-30T06:14:21.539ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "05997a18-0206-47bd-ba8d-9aa71c314c22", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
43672025-04-30T06:14:21.540ZTRACtest_basic: authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet
43682025-04-30T06:14:21.540ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
43692025-04-30T06:14:21.540ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
43702025-04-30T06:14:21.540ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
43712025-04-30T06:14:21.541ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
43722025-04-30T06:14:21.545ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43732025-04-30T06:14:21.551ZDEBGtest_basic: authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet result = Ok(())
43742025-04-30T06:14:21.551ZTRACtest_basic: authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet
43752025-04-30T06:14:21.551ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
43762025-04-30T06:14:21.551ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
43772025-04-30T06:14:21.551ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
43782025-04-30T06:14:21.552ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
43792025-04-30T06:14:21.562ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43802025-04-30T06:14:21.566ZDEBGtest_basic: authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet result = Ok(())
43812025-04-30T06:14:21.567ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
43822025-04-30T06:14:21.567ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
43832025-04-30T06:14:21.567ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
43842025-04-30T06:14:21.628ZDEBGtest_basic: begin collection from Sled Agent sled_agent_url = http://[::1]:42420
43852025-04-30T06:14:21.630ZDEBGtest_basic: client request SledAgent = http://[::1]:42420 body = None method = GET uri = http://[::1]:42420/inventory
43862025-04-30T06:14:21.631ZINFOtest_basic (dropshot): accepted connection local_addr = [::1]:42420 remote_addr = [::1]:57363 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
43872025-04-30T06:14:21.632ZTRACtest_basic (dropshot): incoming request local_addr = [::1]:42420 method = GET remote_addr = [::1]:57363 req_id = cd91c317-0b17-4cb4-8bf5-b0a9feac2686 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
43882025-04-30T06:14:21.634ZINFOtest_basic (dropshot): request completed latency_us = 1759 local_addr = [::1]:42420 method = GET remote_addr = [::1]:57363 req_id = cd91c317-0b17-4cb4-8bf5-b0a9feac2686 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
43892025-04-30T06:14:21.637ZDEBGtest_basic: client response SledAgent = http://[::1]:42420 result = Ok(Response { url: "http://[::1]:42420/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd91c317-0b17-4cb4-8bf5-b0a9feac2686", "content-length": "2769", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
43902025-04-30T06:14:21.650ZDEBGtest_basic: begin collecting all keepers nkeeper_admin_clients = 0
43912025-04-30T06:14:21.650ZDEBGtest_basic: end collecting all keepers nkeeper_admin_clients = 0
43922025-04-30T06:14:21.650ZDEBGtest_basic: finished collection
43932025-04-30T06:14:21.654ZTRACtest_basic: authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory
43942025-04-30T06:14:21.655ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
43952025-04-30T06:14:21.655ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
43962025-04-30T06:14:21.655ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
43972025-04-30T06:14:21.656ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
43982025-04-30T06:14:21.672ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43992025-04-30T06:14:21.682ZDEBGtest_basic: authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory result = Ok(())
44002025-04-30T06:14:21.683ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
44012025-04-30T06:14:21.683ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
44022025-04-30T06:14:21.685ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
44032025-04-30T06:14:21.722ZDEBGtest_basic (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
44042025-04-30T06:14:21.722ZDEBGtest_basic (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
44052025-04-30T06:14:21.723ZDEBGtest_basic (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812
44062025-04-30T06:14:21.724ZTRACtest_basic (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
44072025-04-30T06:14:21.747ZDEBGtest_basic (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_rows = 2 table_name = fields_string
44082025-04-30T06:14:21.768ZTRACtest_basic (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
44092025-04-30T06:14:21.795ZDEBGtest_basic (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_rows = 1 table_name = measurements_i64
44102025-04-30T06:14:21.797ZTRACtest_basic (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
44112025-04-30T06:14:21.820ZWARNtest_basic (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
44122025-04-30T06:14:21.821ZWARNtest_basic: inventory collection failed error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
44132025-04-30T06:14:21.823ZWARNtest_basic (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
44142025-04-30T06:14:21.833ZTRACtest_basic: authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory
44152025-04-30T06:14:21.836ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
44162025-04-30T06:14:21.837ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
44172025-04-30T06:14:21.837ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
44182025-04-30T06:14:21.838ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
44192025-04-30T06:14:21.858ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44202025-04-30T06:14:21.867ZDEBGtest_basic: authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory result = Ok(())
44212025-04-30T06:14:21.867ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
44222025-04-30T06:14:21.867ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
44232025-04-30T06:14:21.869ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
44242025-04-30T06:14:21.873ZDEBGtest_basic: inventory_prune_one: nothing eligible for removal (too few) candidates = [(efcb0ee5-cafa-4692-9b75-fcee4732bdca, 0)]
44252025-04-30T06:14:21.873ZTRACtest_basic: lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
44262025-04-30T06:14:21.875ZTRACtest_basic (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 4f 04 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 O............_mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:57856 req_id = a69ccd62-e992-48d6-af69-07baeafc106a
44272025-04-30T06:14:21.875ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 20228,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:57856 req_id = a69ccd62-e992-48d6-af69-07baeafc106a
44282025-04-30T06:14:21.875ZDEBGtest_basic (store): query key key = _mgs._tcp
44292025-04-30T06:14:21.875ZDEBGtest_basic (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
44302025-04-30T06:14:21.876ZDEBGtest_basic (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:57856 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 37822, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = a69ccd62-e992-48d6-af69-07baeafc106a
44312025-04-30T06:14:21.876ZDEBGtest_basic: lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 37822, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2526, tv_nsec: 675683090 } })
44322025-04-30T06:14:21.876ZTRACtest_basic: lookup_service_targets: looking up SRV target
44332025-04-30T06:14:21.877ZTRACtest_basic (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 9c 8e 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:60379 req_id = 0c5ef141-cbc9-4f2f-b7fb-c96fb3ab2490
44342025-04-30T06:14:21.877ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 40078,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60379 req_id = 0c5ef141-cbc9-4f2f-b7fb-c96fb3ab2490
44352025-04-30T06:14:21.877ZDEBGtest_basic (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
44362025-04-30T06:14:21.877ZDEBGtest_basic (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:60379 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 0c5ef141-cbc9-4f2f-b7fb-c96fb3ab2490
44372025-04-30T06:14:21.945ZTRACtest_basic: lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
44382025-04-30T06:14:21.945ZTRACtest_basic (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 6a 2e 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c j............_cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:54878 req_id = c21e7a41-b85f-407e-a799-f6f01b5f40e4
44392025-04-30T06:14:21.945ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 27182,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54878 req_id = c21e7a41-b85f-407e-a799-f6f01b5f40e4
44402025-04-30T06:14:21.946ZDEBGtest_basic (store): query key key = _clickhouse-admin-keeper._tcp
44412025-04-30T06:14:21.946ZERROtest_basic (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 27182, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:54878 req_id = c21e7a41-b85f-407e-a799-f6f01b5f40e4
44422025-04-30T06:14:21.946ZDEBGtest_basic: begin collection
44432025-04-30T06:14:21.946ZDEBGtest_basic: begin collection from MGS mgs_url = http://[::1]:37822
44442025-04-30T06:14:21.946ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/ignition
44452025-04-30T06:14:21.948ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "2df03a60-b57f-4945-b40e-4e9106587895", "content-length": "835", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
44462025-04-30T06:14:21.948ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0
44472025-04-30T06:14:21.950ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a82269d-57bd-4eb7-948b-213268cf6c61", "content-length": "734", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
44482025-04-30T06:14:21.950ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=0
44492025-04-30T06:14:21.952ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "28b1bb69-0d48-4753-a567-82564f005882", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
44502025-04-30T06:14:21.952ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=1
44512025-04-30T06:14:21.954ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ec69c97a-cb05-47f7-9329-21a77f6e7cbb", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
44522025-04-30T06:14:21.954ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=0
44532025-04-30T06:14:21.957ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8f2e5c60-a460-4a46-96fd-98fa351edae2", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
44542025-04-30T06:14:21.957ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=1
44552025-04-30T06:14:21.960ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6670f821-ac88-4956-beba-48290791b58a", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
44562025-04-30T06:14:21.960ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=0
44572025-04-30T06:14:21.964ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "68c2f1c0-0918-47f1-b844-9a17ee186b69", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
44582025-04-30T06:14:21.964ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=1
44592025-04-30T06:14:21.966ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "eafa13b4-4157-4b38-a592-6182d863d0ad", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
44602025-04-30T06:14:21.967ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cmpa
44612025-04-30T06:14:21.968ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a97310f7-a1e7-4313-a5e4-c3a7e5659bda", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
44622025-04-30T06:14:21.968ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
44632025-04-30T06:14:21.969ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c09a51b1-476b-4886-936f-1b0e2586aba6", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
44642025-04-30T06:14:21.969ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
44652025-04-30T06:14:21.973ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7fb5627b-6958-415c-8f8c-5739012f5c7f", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
44662025-04-30T06:14:21.973ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
44672025-04-30T06:14:21.973ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "60c2c1ed-e840-44ec-8f9f-7f14d452965a", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
44682025-04-30T06:14:21.973ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1
44692025-04-30T06:14:21.975ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "96aeb729-cbe9-457b-9306-c973c8f7e8ae", "content-length": "734", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
44702025-04-30T06:14:21.975ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=0
44712025-04-30T06:14:21.981ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d742ecc7-b322-4968-8a8a-6f403d8b2d07", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
44722025-04-30T06:14:21.981ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=1
44732025-04-30T06:14:21.988ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6f81be20-a908-4e1c-b03b-44145fa22313", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
44742025-04-30T06:14:21.989ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=0
44752025-04-30T06:14:21.995ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2d447991-9b4e-4a93-aab3-a46db47833bc", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
44762025-04-30T06:14:21.996ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=1
44772025-04-30T06:14:22.008ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "764d7f33-f5c8-4dbb-b20f-be6dadfbb295", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
44782025-04-30T06:14:22.012ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=0
44792025-04-30T06:14:22.020ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5e8256fd-2335-4d6f-b884-093e63b798e0", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
44802025-04-30T06:14:22.020ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=1
44812025-04-30T06:14:22.048ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "54a8c0c0-bfa1-4574-9c60-db1c42302e4e", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
44822025-04-30T06:14:22.048ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cmpa
44832025-04-30T06:14:22.058ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "81230631-8696-457b-9db8-b42e2b3108c0", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:21 GMT"} })
44842025-04-30T06:14:22.058ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
44852025-04-30T06:14:22.062ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d310c1b7-fa42-4b6c-a7dc-465d92221760", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
44862025-04-30T06:14:22.062ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
44872025-04-30T06:14:22.063ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "43be5cb0-9f1d-4808-93f2-36475f8b21e7", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
44882025-04-30T06:14:22.063ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
44892025-04-30T06:14:22.065ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c2772a14-e54f-4c76-aaae-cd1910ad9f32", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
44902025-04-30T06:14:22.065ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0
44912025-04-30T06:14:22.067ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0b209f67-96eb-41d1-852c-c0bf8df60423", "content-length": "723", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
44922025-04-30T06:14:22.068ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=0
44932025-04-30T06:14:22.072ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "351cd0b3-76fc-40b4-98d8-4cc70e49e176", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
44942025-04-30T06:14:22.076ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=1
44952025-04-30T06:14:22.156ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c014c26e-3e49-4a7f-8a74-f13a6d7f941b", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
44962025-04-30T06:14:22.156ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=0
44972025-04-30T06:14:22.179ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7412055b-ad30-4371-9b6f-a186686c642a", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
44982025-04-30T06:14:22.180ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=1
44992025-04-30T06:14:22.182ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "347b288a-f6e4-4123-953c-35961e0a32f4", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
45002025-04-30T06:14:22.182ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=0
45012025-04-30T06:14:22.183ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "638ea92d-95f0-422a-a8f2-b202e0f5625f", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
45022025-04-30T06:14:22.184ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=1
45032025-04-30T06:14:22.187ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2c135655-8aa1-48ac-9672-405887a80f49", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
45042025-04-30T06:14:22.187ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cmpa
45052025-04-30T06:14:22.187ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4abb1f43-5b61-412e-88a3-f1c09c71414f", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
45062025-04-30T06:14:22.188ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
45072025-04-30T06:14:22.191ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5a58ca3e-f4dd-4100-b4a5-84425f30dcdf", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
45082025-04-30T06:14:22.194ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
45092025-04-30T06:14:22.195ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "98e55dae-a8e5-4a7c-ae5b-d55a1ec0f7dd", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
45102025-04-30T06:14:22.195ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
45112025-04-30T06:14:22.196ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "446ff13d-9dbf-48e7-8f20-b004035987ba", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
45122025-04-30T06:14:22.196ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1
45132025-04-30T06:14:22.197ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "69979613-2ad9-426d-ab00-0780dffb3d57", "content-length": "723", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
45142025-04-30T06:14:22.197ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=0
45152025-04-30T06:14:22.199ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "91dcc6c8-f5b5-427a-b912-617b54373bb3", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
45162025-04-30T06:14:22.199ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=1
45172025-04-30T06:14:22.201ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "27c6bde7-82c1-4e9e-bf2b-a63473c421f1", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
45182025-04-30T06:14:22.201ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=0
45192025-04-30T06:14:22.215ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c39800c7-2794-4385-901b-7a97fc621de7", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
45202025-04-30T06:14:22.216ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=1
45212025-04-30T06:14:22.218ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1d8c1906-32e8-4515-b4a6-9b86ef479ddb", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
45222025-04-30T06:14:22.219ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=0
45232025-04-30T06:14:22.225ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "afe78ee0-e282-4f40-a1f0-b6a537ae5251", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
45242025-04-30T06:14:22.227ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=1
45252025-04-30T06:14:22.229ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fc372c47-0a75-4a40-aaa9-8eddfca53335", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
45262025-04-30T06:14:22.229ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cmpa
45272025-04-30T06:14:22.230ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ac9feef3-2b38-4731-b413-28daba6c01d4", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
45282025-04-30T06:14:22.230ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
45292025-04-30T06:14:22.231ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "77880731-ce12-4fa8-952d-b46029c17223", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
45302025-04-30T06:14:22.231ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
45312025-04-30T06:14:22.235ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0abc6080-9db2-49a6-a47c-cd4c51942ee9", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
45322025-04-30T06:14:22.235ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
45332025-04-30T06:14:22.237ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fb2e75e2-7682-46ed-a05f-bc78b441db44", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
45342025-04-30T06:14:22.238ZTRACtest_basic: authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet
45352025-04-30T06:14:22.238ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
45362025-04-30T06:14:22.238ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
45372025-04-30T06:14:22.238ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
45382025-04-30T06:14:22.239ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
45392025-04-30T06:14:22.243ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45402025-04-30T06:14:22.251ZDEBGtest_basic: authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet result = Ok(())
45412025-04-30T06:14:22.251ZTRACtest_basic: authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet
45422025-04-30T06:14:22.251ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
45432025-04-30T06:14:22.251ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
45442025-04-30T06:14:22.251ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
45452025-04-30T06:14:22.252ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
45462025-04-30T06:14:22.256ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45472025-04-30T06:14:22.269ZDEBGtest_basic: authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet result = Ok(())
45482025-04-30T06:14:22.269ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
45492025-04-30T06:14:22.269ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
45502025-04-30T06:14:22.271ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
45512025-04-30T06:14:22.343ZDEBGtest_basic: begin collection from Sled Agent sled_agent_url = http://[::1]:42420
45522025-04-30T06:14:22.343ZDEBGtest_basic: client request SledAgent = http://[::1]:42420 body = None method = GET uri = http://[::1]:42420/inventory
45532025-04-30T06:14:22.344ZINFOtest_basic (dropshot): accepted connection local_addr = [::1]:42420 remote_addr = [::1]:60654 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
45542025-04-30T06:14:22.345ZTRACtest_basic (dropshot): incoming request local_addr = [::1]:42420 method = GET remote_addr = [::1]:60654 req_id = bb1091ab-3b22-4182-8c24-d9cc1ad18d3b sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
45552025-04-30T06:14:22.345ZINFOtest_basic (dropshot): request completed latency_us = 799 local_addr = [::1]:42420 method = GET remote_addr = [::1]:60654 req_id = bb1091ab-3b22-4182-8c24-d9cc1ad18d3b response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
45562025-04-30T06:14:22.346ZDEBGtest_basic: client response SledAgent = http://[::1]:42420 result = Ok(Response { url: "http://[::1]:42420/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb1091ab-3b22-4182-8c24-d9cc1ad18d3b", "content-length": "2769", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
45572025-04-30T06:14:22.348ZDEBGtest_basic: begin collecting all keepers nkeeper_admin_clients = 0
45582025-04-30T06:14:22.348ZDEBGtest_basic: end collecting all keepers nkeeper_admin_clients = 0
45592025-04-30T06:14:22.348ZDEBGtest_basic: finished collection
45602025-04-30T06:14:22.354ZTRACtest_basic: authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory
45612025-04-30T06:14:22.354ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
45622025-04-30T06:14:22.354ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
45632025-04-30T06:14:22.354ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
45642025-04-30T06:14:22.355ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
45652025-04-30T06:14:22.357ZDEBGtest_basic (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
45662025-04-30T06:14:22.357ZDEBGtest_basic (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
45672025-04-30T06:14:22.359ZTRACtest_basic (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
45682025-04-30T06:14:22.360ZDEBGtest_basic (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
45692025-04-30T06:14:22.360ZTRACtest_basic (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
45702025-04-30T06:14:22.360ZTRACtest_basic (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
45712025-04-30T06:14:22.361ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45722025-04-30T06:14:22.371ZDEBGtest_basic: authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory result = Ok(())
45732025-04-30T06:14:22.373ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
45742025-04-30T06:14:22.373ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
45752025-04-30T06:14:22.376ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
45762025-04-30T06:14:22.439ZWARNtest_basic (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
45772025-04-30T06:14:22.441ZWARNtest_basic: inventory collection failed error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
45782025-04-30T06:14:22.448ZTRACtest_basic: authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory
45792025-04-30T06:14:22.448ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
45802025-04-30T06:14:22.448ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
45812025-04-30T06:14:22.448ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
45822025-04-30T06:14:22.450ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
45832025-04-30T06:14:22.460ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45842025-04-30T06:14:22.473ZDEBGtest_basic: authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory result = Ok(())
45852025-04-30T06:14:22.473ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
45862025-04-30T06:14:22.473ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
45872025-04-30T06:14:22.474ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
45882025-04-30T06:14:22.484ZDEBGtest_basic: inventory_prune_one: nothing eligible for removal (too few) candidates = [(efcb0ee5-cafa-4692-9b75-fcee4732bdca, 0)]
45892025-04-30T06:14:22.484ZTRACtest_basic: lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
45902025-04-30T06:14:22.486ZTRACtest_basic (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 8a c3 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:58769 req_id = 3355a7be-5ca6-4b8b-9324-98a931d68709
45912025-04-30T06:14:22.486ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 35523,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:58769 req_id = 3355a7be-5ca6-4b8b-9324-98a931d68709
45922025-04-30T06:14:22.487ZDEBGtest_basic (store): query key key = _mgs._tcp
45932025-04-30T06:14:22.487ZDEBGtest_basic (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
45942025-04-30T06:14:22.487ZDEBGtest_basic (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:58769 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 37822, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 3355a7be-5ca6-4b8b-9324-98a931d68709
45952025-04-30T06:14:22.488ZDEBGtest_basic: lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 37822, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2527, tv_nsec: 287054538 } })
45962025-04-30T06:14:22.488ZTRACtest_basic: lookup_service_targets: looking up SRV target
45972025-04-30T06:14:22.488ZTRACtest_basic (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 96 f6 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:58549 req_id = 2593f364-7020-4473-adb8-2c7888b67599
45982025-04-30T06:14:22.488ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 38646,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:58549 req_id = 2593f364-7020-4473-adb8-2c7888b67599
45992025-04-30T06:14:22.489ZDEBGtest_basic (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
46002025-04-30T06:14:22.489ZDEBGtest_basic (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:58549 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 2593f364-7020-4473-adb8-2c7888b67599
46012025-04-30T06:14:22.551ZTRACtest_basic: lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
46022025-04-30T06:14:22.552ZTRACtest_basic (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 4d df 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c M............_cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:58916 req_id = 9e72b4fa-6371-45c0-ab6f-9f0d26b619ff
46032025-04-30T06:14:22.552ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 19935,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:58916 req_id = 9e72b4fa-6371-45c0-ab6f-9f0d26b619ff
46042025-04-30T06:14:22.552ZDEBGtest_basic (store): query key key = _clickhouse-admin-keeper._tcp
46052025-04-30T06:14:22.553ZERROtest_basic (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 19935, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:58916 req_id = 9e72b4fa-6371-45c0-ab6f-9f0d26b619ff
46062025-04-30T06:14:22.553ZDEBGtest_basic: begin collection
46072025-04-30T06:14:22.553ZDEBGtest_basic: begin collection from MGS mgs_url = http://[::1]:37822
46082025-04-30T06:14:22.553ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/ignition
46092025-04-30T06:14:22.555ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "f6af980b-c5d2-4eb5-8a47-6f17af7a353b", "content-length": "835", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
46102025-04-30T06:14:22.555ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0
46112025-04-30T06:14:22.557ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "aa0b32c4-0ff4-4637-bb8c-06441f6d6626", "content-length": "734", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
46122025-04-30T06:14:22.557ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=0
46132025-04-30T06:14:22.563ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4f8379f7-ebdd-40bd-a04d-be18f6693cfc", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
46142025-04-30T06:14:22.563ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=1
46152025-04-30T06:14:22.567ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "436bf133-6ed5-4f24-9e05-fbd1b18851e1", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
46162025-04-30T06:14:22.567ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=0
46172025-04-30T06:14:22.572ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0af2ea83-dbeb-4b1b-9af5-cea7ca9a6b6d", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
46182025-04-30T06:14:22.578ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=1
46192025-04-30T06:14:22.589ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5b860864-c573-4347-b42b-77becec54b99", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
46202025-04-30T06:14:22.590ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=0
46212025-04-30T06:14:22.593ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "61f3c0ca-54c9-49c5-b545-f5a7dc8b1b80", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
46222025-04-30T06:14:22.593ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=1
46232025-04-30T06:14:22.597ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bf17ef05-91b7-49d1-a70b-bf2583171916", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
46242025-04-30T06:14:22.598ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cmpa
46252025-04-30T06:14:22.599ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a8fb6e34-d4db-4073-918c-323298a6831b", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
46262025-04-30T06:14:22.599ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
46272025-04-30T06:14:22.603ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b45a09c4-edad-487e-a345-9e5e53bfdfd4", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
46282025-04-30T06:14:22.604ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
46292025-04-30T06:14:22.606ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2f86e119-4671-495d-8bd8-12582085ed70", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
46302025-04-30T06:14:22.606ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
46312025-04-30T06:14:22.608ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c68f0bde-172d-472c-ad0d-e2d5d0392707", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
46322025-04-30T06:14:22.608ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1
46332025-04-30T06:14:22.610ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "92d9ce39-1457-4d57-9d37-aa5e7e3cd386", "content-length": "734", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
46342025-04-30T06:14:22.610ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=0
46352025-04-30T06:14:22.614ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7752bf76-54a1-4555-97d7-a314761f8bf8", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
46362025-04-30T06:14:22.615ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=1
46372025-04-30T06:14:22.617ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d815c0a2-882b-47de-bb39-ce66dd3afbf9", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
46382025-04-30T06:14:22.617ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=0
46392025-04-30T06:14:22.620ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ac35064d-3a04-4439-a94f-86ef224a2113", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
46402025-04-30T06:14:22.620ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=1
46412025-04-30T06:14:22.623ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cf61bcc2-40c0-4a4f-8b28-445bf69991f1", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
46422025-04-30T06:14:22.623ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=0
46432025-04-30T06:14:22.627ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "baabcc9d-4fac-4935-a704-f7755ac48149", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
46442025-04-30T06:14:22.627ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=1
46452025-04-30T06:14:22.629ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a60dc3d4-80f1-495c-b10d-9032ef200a72", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
46462025-04-30T06:14:22.629ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cmpa
46472025-04-30T06:14:22.630ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0d9fb95-6859-4008-8117-cffdefd8dae2", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
46482025-04-30T06:14:22.630ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
46492025-04-30T06:14:22.631ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f5f242a9-64e2-41d2-8e85-c2688645e80a", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
46502025-04-30T06:14:22.631ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
46512025-04-30T06:14:22.632ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "64c78c33-3701-4b3d-a309-c3d9b00a1e1a", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
46522025-04-30T06:14:22.633ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
46532025-04-30T06:14:22.633ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "be83b478-0799-4de5-be4a-8109c147e132", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
46542025-04-30T06:14:22.634ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0
46552025-04-30T06:14:22.635ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c7b90784-91b6-4fa7-8c42-eb164fbb2048", "content-length": "723", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
46562025-04-30T06:14:22.635ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=0
46572025-04-30T06:14:22.638ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c3299de9-cba2-450b-bb1d-4043b0de179e", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
46582025-04-30T06:14:22.639ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=1
46592025-04-30T06:14:22.641ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "273ee9fc-3e74-483a-9a78-5254c3a23ab1", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
46602025-04-30T06:14:22.641ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=0
46612025-04-30T06:14:22.644ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d7aec7db-6393-4f3e-9ec4-804576dc46ac", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
46622025-04-30T06:14:22.644ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=1
46632025-04-30T06:14:22.647ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f4606748-6edd-44ca-bf76-13a6f882d54d", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
46642025-04-30T06:14:22.647ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=0
46652025-04-30T06:14:22.650ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "52e9cf82-42b8-4687-98b1-2f3917aa78e3", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
46662025-04-30T06:14:22.650ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=1
46672025-04-30T06:14:22.661ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a390d4b3-b77f-4af7-a982-2854e73f03b2", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
46682025-04-30T06:14:22.662ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cmpa
46692025-04-30T06:14:22.664ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cea271cd-843d-4584-a213-44b49291ba60", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
46702025-04-30T06:14:22.665ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
46712025-04-30T06:14:22.666ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "31042418-6067-44a1-8883-2ec72c7744f3", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
46722025-04-30T06:14:22.666ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
46732025-04-30T06:14:22.667ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9d771bb4-27ee-4ad1-9e9d-abb23f167f13", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
46742025-04-30T06:14:22.667ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
46752025-04-30T06:14:22.668ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a7b8fe7-6d1a-4a20-be45-ea9230d3d0e7", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
46762025-04-30T06:14:22.669ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1
46772025-04-30T06:14:22.672ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "792d8aa7-e63f-4eb4-85d2-ab8dbe522d4d", "content-length": "723", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
46782025-04-30T06:14:22.673ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=0
46792025-04-30T06:14:22.676ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "53dc063c-75d8-4900-bf2c-febff524f78a", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
46802025-04-30T06:14:22.676ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=1
46812025-04-30T06:14:22.678ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f9e1f6dd-5de4-4edf-aa04-f4ab77d32e2d", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
46822025-04-30T06:14:22.679ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=0
46832025-04-30T06:14:22.682ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2502c0be-6a77-42a0-bd44-5e1f868ece8f", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
46842025-04-30T06:14:22.683ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=1
46852025-04-30T06:14:22.691ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "91e1866b-4001-4a74-b6ef-35d0583f576c", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
46862025-04-30T06:14:22.691ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=0
46872025-04-30T06:14:22.696ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4bb96cb0-25c9-42c1-929e-40b77a61ef99", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
46882025-04-30T06:14:22.696ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=1
46892025-04-30T06:14:22.700ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "11dff9a7-466c-4122-ba69-4d0b75ebfd96", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
46902025-04-30T06:14:22.701ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cmpa
46912025-04-30T06:14:22.702ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4ac0cf59-9f52-4626-9f9b-b6a442faeea0", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
46922025-04-30T06:14:22.702ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
46932025-04-30T06:14:22.703ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "29ce4daa-17ab-4975-b34e-623a8e58e401", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
46942025-04-30T06:14:22.703ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
46952025-04-30T06:14:22.706ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "baf02ee6-a15c-4953-9fe1-916e68a3e0f2", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
46962025-04-30T06:14:22.706ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
46972025-04-30T06:14:22.708ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0a52f1d-e0fb-4a19-99fb-0fc837778b31", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
46982025-04-30T06:14:22.708ZTRACtest_basic: authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet
46992025-04-30T06:14:22.708ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
47002025-04-30T06:14:22.708ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
47012025-04-30T06:14:22.708ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
47022025-04-30T06:14:22.710ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
47032025-04-30T06:14:22.713ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47042025-04-30T06:14:22.719ZDEBGtest_basic: authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet result = Ok(())
47052025-04-30T06:14:22.719ZTRACtest_basic: authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet
47062025-04-30T06:14:22.719ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
47072025-04-30T06:14:22.719ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
47082025-04-30T06:14:22.719ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
47092025-04-30T06:14:22.720ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
47102025-04-30T06:14:22.720ZDEBGtest_basic (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
47112025-04-30T06:14:22.720ZDEBGtest_basic (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
47122025-04-30T06:14:22.721ZDEBGtest_basic (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812
47132025-04-30T06:14:22.722ZTRACtest_basic (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
47142025-04-30T06:14:22.724ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47152025-04-30T06:14:22.729ZDEBGtest_basic: authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet result = Ok(())
47162025-04-30T06:14:22.729ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
47172025-04-30T06:14:22.729ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
47182025-04-30T06:14:22.730ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
47192025-04-30T06:14:22.731ZDEBGtest_basic (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_rows = 2 table_name = fields_string
47202025-04-30T06:14:22.731ZTRACtest_basic (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
47212025-04-30T06:14:22.779ZDEBGtest_basic: begin collection from Sled Agent sled_agent_url = http://[::1]:42420
47222025-04-30T06:14:22.779ZDEBGtest_basic: client request SledAgent = http://[::1]:42420 body = None method = GET uri = http://[::1]:42420/inventory
47232025-04-30T06:14:22.779ZDEBGtest_basic (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_rows = 1 table_name = measurements_i64
47242025-04-30T06:14:22.779ZTRACtest_basic (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
47252025-04-30T06:14:22.780ZINFOtest_basic (dropshot): accepted connection local_addr = [::1]:42420 remote_addr = [::1]:43716 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
47262025-04-30T06:14:22.780ZTRACtest_basic (dropshot): incoming request local_addr = [::1]:42420 method = GET remote_addr = [::1]:43716 req_id = bef47229-f076-447c-8c91-39bf73d80000 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
47272025-04-30T06:14:22.781ZINFOtest_basic (dropshot): request completed latency_us = 704 local_addr = [::1]:42420 method = GET remote_addr = [::1]:43716 req_id = bef47229-f076-447c-8c91-39bf73d80000 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
47282025-04-30T06:14:22.782ZDEBGtest_basic: client response SledAgent = http://[::1]:42420 result = Ok(Response { url: "http://[::1]:42420/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "bef47229-f076-447c-8c91-39bf73d80000", "content-length": "2769", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
47292025-04-30T06:14:22.785ZDEBGtest_basic: begin collecting all keepers nkeeper_admin_clients = 0
47302025-04-30T06:14:22.785ZDEBGtest_basic: end collecting all keepers nkeeper_admin_clients = 0
47312025-04-30T06:14:22.785ZDEBGtest_basic: finished collection
47322025-04-30T06:14:22.790ZTRACtest_basic: authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory
47332025-04-30T06:14:22.790ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
47342025-04-30T06:14:22.790ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
47352025-04-30T06:14:22.791ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
47362025-04-30T06:14:22.792ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
47372025-04-30T06:14:22.796ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47382025-04-30T06:14:22.802ZDEBGtest_basic: authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory result = Ok(())
47392025-04-30T06:14:22.802ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
47402025-04-30T06:14:22.802ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
47412025-04-30T06:14:22.803ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
47422025-04-30T06:14:22.821ZWARNtest_basic (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
47432025-04-30T06:14:22.836ZWARNtest_basic (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
47442025-04-30T06:14:22.837ZWARNtest_basic: inventory collection failed error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
47452025-04-30T06:14:22.846ZTRACtest_basic: authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory
47462025-04-30T06:14:22.846ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
47472025-04-30T06:14:22.846ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
47482025-04-30T06:14:22.846ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
47492025-04-30T06:14:22.850ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
47502025-04-30T06:14:22.855ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47512025-04-30T06:14:22.862ZDEBGtest_basic: authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory result = Ok(())
47522025-04-30T06:14:22.862ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
47532025-04-30T06:14:22.862ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
47542025-04-30T06:14:22.863ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
47552025-04-30T06:14:22.866ZDEBGtest_basic: inventory_prune_one: nothing eligible for removal (too few) candidates = [(efcb0ee5-cafa-4692-9b75-fcee4732bdca, 0)]
47562025-04-30T06:14:22.867ZTRACtest_basic: lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
47572025-04-30T06:14:22.868ZTRACtest_basic (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 22 5a 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 "Z..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:59288 req_id = e2d8e5fc-207c-48b0-95de-9ae63b41a949
47582025-04-30T06:14:22.868ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 8794,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:59288 req_id = e2d8e5fc-207c-48b0-95de-9ae63b41a949
47592025-04-30T06:14:22.869ZDEBGtest_basic (store): query key key = _mgs._tcp
47602025-04-30T06:14:22.869ZDEBGtest_basic (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
47612025-04-30T06:14:22.869ZDEBGtest_basic (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:59288 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 37822, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = e2d8e5fc-207c-48b0-95de-9ae63b41a949
47622025-04-30T06:14:22.870ZDEBGtest_basic: lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 37822, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2527, tv_nsec: 669263544 } })
47632025-04-30T06:14:22.870ZTRACtest_basic: lookup_service_targets: looking up SRV target
47642025-04-30T06:14:22.872ZTRACtest_basic (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 6a 86 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e j...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:62136 req_id = aba64933-42be-40a8-b5a3-eef12e3c2423
47652025-04-30T06:14:22.872ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 27270,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62136 req_id = aba64933-42be-40a8-b5a3-eef12e3c2423
47662025-04-30T06:14:22.872ZDEBGtest_basic (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
47672025-04-30T06:14:22.872ZDEBGtest_basic (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:62136 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = aba64933-42be-40a8-b5a3-eef12e3c2423
47682025-04-30T06:14:22.958ZTRACtest_basic: lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
47692025-04-30T06:14:22.961ZTRACtest_basic (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 3b 00 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ;............_cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:62011 req_id = a299e65d-30e2-4434-b56b-80cf6b76a945
47702025-04-30T06:14:22.963ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 15104,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62011 req_id = a299e65d-30e2-4434-b56b-80cf6b76a945
47712025-04-30T06:14:22.964ZDEBGtest_basic (store): query key key = _clickhouse-admin-keeper._tcp
47722025-04-30T06:14:22.964ZERROtest_basic (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 15104, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:62011 req_id = a299e65d-30e2-4434-b56b-80cf6b76a945
47732025-04-30T06:14:22.964ZDEBGtest_basic: begin collection
47742025-04-30T06:14:22.964ZDEBGtest_basic: begin collection from MGS mgs_url = http://[::1]:37822
47752025-04-30T06:14:22.964ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/ignition
47762025-04-30T06:14:22.966ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "ee8d3e89-b299-4bd2-8f3a-d974d946b030", "content-length": "835", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
47772025-04-30T06:14:22.967ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0
47782025-04-30T06:14:22.968ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "42041789-99f4-4a62-9266-1cdd6a1b3c86", "content-length": "734", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
47792025-04-30T06:14:22.969ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=0
47802025-04-30T06:14:22.971ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "20991756-a6b7-4de9-8daa-67e7ce9474d5", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
47812025-04-30T06:14:22.976ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=1
47822025-04-30T06:14:22.980ZINFOtest_basic (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
47832025-04-30T06:14:22.982ZDEBGtest_basic (oximeter): client request body = None method = GET uri = http://127.0.0.1:34211/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
47842025-04-30T06:14:22.983ZTRACtest_basic (dropshot_internal): incoming request local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = a96e98b1-77dc-4525-ab4b-bf341d0ae9ca uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
47852025-04-30T06:14:22.983ZTRACtest_basic (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = a96e98b1-77dc-4525-ab4b-bf341d0ae9ca resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
47862025-04-30T06:14:22.983ZDEBGtest_basic (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = a96e98b1-77dc-4525-ab4b-bf341d0ae9ca roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
47872025-04-30T06:14:22.985ZDEBGtest_basic (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = a96e98b1-77dc-4525-ab4b-bf341d0ae9ca resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
47882025-04-30T06:14:22.988ZINFOtest_basic (dropshot_internal): request completed latency_us = 5744 local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = a96e98b1-77dc-4525-ab4b-bf341d0ae9ca response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
47892025-04-30T06:14:22.989ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "76906314-4648-45eb-b9dc-d7b190dd85b5", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
47902025-04-30T06:14:22.989ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=0
47912025-04-30T06:14:22.989ZDEBGtest_basic (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:34211/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "a96e98b1-77dc-4525-ab4b-bf341d0ae9ca", "content-length": "405", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
47922025-04-30T06:14:22.989ZDEBGtest_basic (oximeter): client request body = None method = GET uri = http://127.0.0.1:34211/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
47932025-04-30T06:14:22.990ZTRACtest_basic (dropshot_internal): incoming request local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 8b80b22b-fc22-45cb-b033-ca563451ac27 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
47942025-04-30T06:14:22.991ZTRACtest_basic (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 8b80b22b-fc22-45cb-b033-ca563451ac27 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
47952025-04-30T06:14:22.991ZDEBGtest_basic (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 8b80b22b-fc22-45cb-b033-ca563451ac27 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
47962025-04-30T06:14:22.993ZDEBGtest_basic (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 8b80b22b-fc22-45cb-b033-ca563451ac27 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
47972025-04-30T06:14:22.997ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "26e7c9c6-9f02-41d5-b321-ae6adbf57235", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
47982025-04-30T06:14:22.997ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=1
47992025-04-30T06:14:22.997ZINFOtest_basic (dropshot_internal): request completed latency_us = 7136 local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 8b80b22b-fc22-45cb-b033-ca563451ac27 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
48002025-04-30T06:14:22.998ZDEBGtest_basic (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:34211/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "8b80b22b-fc22-45cb-b033-ca563451ac27", "content-length": "29", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
48012025-04-30T06:14:22.998ZTRACtest_basic (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 088ac96c-981b-4054-bea6-d8ab23ae1366
48022025-04-30T06:14:22.998ZTRACtest_basic (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
48032025-04-30T06:14:22.998ZINFOtest_basic (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
48042025-04-30T06:14:23.001ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0cdfd318-229b-48bf-9d41-d22df3c831f5", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
48052025-04-30T06:14:23.001ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=0
48062025-04-30T06:14:23.002ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "004b4246-47e1-4fac-a089-458e12c0c515", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
48072025-04-30T06:14:23.003ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=1
48082025-04-30T06:14:23.004ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "943da030-6e42-41ab-b0e9-ef0303bdbb83", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
48092025-04-30T06:14:23.004ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cmpa
48102025-04-30T06:14:23.005ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d0c9aaa2-87a7-4d53-b5f7-dbb8d082682d", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
48112025-04-30T06:14:23.005ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
48122025-04-30T06:14:23.008ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c305c678-597a-4369-8ce3-6abd90d3fed4", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
48132025-04-30T06:14:23.008ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
48142025-04-30T06:14:23.009ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7db24013-2181-4f5d-a667-12c0496f3c28", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
48152025-04-30T06:14:23.009ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
48162025-04-30T06:14:23.010ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ba081a75-115f-4a64-9f1a-f3184d4c9d9b", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
48172025-04-30T06:14:23.010ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1
48182025-04-30T06:14:23.011ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3e67e3d2-a7c4-4e41-8d97-f227706d41ff", "content-length": "734", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
48192025-04-30T06:14:23.011ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=0
48202025-04-30T06:14:23.013ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e7122bcb-f051-4a79-aa1a-b453b33d3415", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
48212025-04-30T06:14:23.013ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=1
48222025-04-30T06:14:23.015ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a8c894eb-b196-47ea-94e4-a7dae399ab44", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
48232025-04-30T06:14:23.015ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=0
48242025-04-30T06:14:23.017ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2465915c-4225-42f5-b26f-85aeed7773f8", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
48252025-04-30T06:14:23.017ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=1
48262025-04-30T06:14:23.019ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7128e29a-a379-49e9-a58a-7f4752c313b5", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
48272025-04-30T06:14:23.019ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=0
48282025-04-30T06:14:23.021ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "42edb8af-a5fb-442b-b64a-83cd17ee3b4a", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
48292025-04-30T06:14:23.021ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=1
48302025-04-30T06:14:23.024ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ab374697-e101-48eb-8207-4e4f0478e821", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
48312025-04-30T06:14:23.025ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cmpa
48322025-04-30T06:14:23.028ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6aa42640-c1ce-471c-a259-202758a9faa0", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
48332025-04-30T06:14:23.028ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
48342025-04-30T06:14:23.028ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b3faf41e-f5cf-43ea-a7f5-269060efdb6a", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
48352025-04-30T06:14:23.028ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
48362025-04-30T06:14:23.030ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "06b273f5-419e-4383-9209-fe5a439e46fa", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
48372025-04-30T06:14:23.031ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
48382025-04-30T06:14:23.046ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "445450b0-d891-436a-bb90-35b60fdbec42", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
48392025-04-30T06:14:23.047ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0
48402025-04-30T06:14:23.049ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0688c1ed-cbab-484a-b5f3-8e35ad663656", "content-length": "723", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
48412025-04-30T06:14:23.049ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=0
48422025-04-30T06:14:23.053ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a26ab9c-fc55-43e2-9656-49c744ce524f", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
48432025-04-30T06:14:23.053ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=1
48442025-04-30T06:14:23.057ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d65fc5ed-845f-4540-a0f2-d522335c0255", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:22 GMT"} })
48452025-04-30T06:14:23.058ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=0
48462025-04-30T06:14:23.069ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "92659596-592c-4645-b9dd-1189f1875bd3", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
48472025-04-30T06:14:23.069ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=1
48482025-04-30T06:14:23.073ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "99797dda-7e4e-41c9-8aad-d32f2acef066", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
48492025-04-30T06:14:23.073ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=0
48502025-04-30T06:14:23.074ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bc5a584e-7658-4ef9-ad1f-c4d543d2976f", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
48512025-04-30T06:14:23.074ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=1
48522025-04-30T06:14:23.089ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bd4a78ee-a390-4b8d-8fda-c8e61b989eee", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
48532025-04-30T06:14:23.089ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cmpa
48542025-04-30T06:14:23.101ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a01e2b70-6a21-4839-b53e-33ca58f082a0", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
48552025-04-30T06:14:23.102ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
48562025-04-30T06:14:23.115ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4e66c9f7-e4ec-4976-b5a9-85fe5bbb978f", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
48572025-04-30T06:14:23.119ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
48582025-04-30T06:14:23.127ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "853a80b3-d732-4870-bcf0-620b4bed9375", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
48592025-04-30T06:14:23.127ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
48602025-04-30T06:14:23.136ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "edb234b8-24a8-4756-90f5-1e39305f5d7a", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
48612025-04-30T06:14:23.136ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1
48622025-04-30T06:14:23.138ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e747f17e-8a82-4246-926d-3b39ffe0aa4b", "content-length": "723", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
48632025-04-30T06:14:23.138ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=0
48642025-04-30T06:14:23.140ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c8fb4465-dcf2-4ca5-a8af-62d1645f2942", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
48652025-04-30T06:14:23.141ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=1
48662025-04-30T06:14:23.143ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "68a97181-e857-44f7-85dc-9dbe2ada9fb0", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
48672025-04-30T06:14:23.143ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=0
48682025-04-30T06:14:23.144ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "933cefd7-965b-43e5-a09f-21ba47901b6e", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
48692025-04-30T06:14:23.145ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=1
48702025-04-30T06:14:23.147ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c90553a0-622f-458f-a1f3-40962ef94a86", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
48712025-04-30T06:14:23.148ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=0
48722025-04-30T06:14:23.150ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1dd39bc7-dc6d-4f9f-9d98-62e870c2bb67", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
48732025-04-30T06:14:23.150ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=1
48742025-04-30T06:14:23.153ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "beec1201-393c-4874-a5b2-edac85d5472e", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
48752025-04-30T06:14:23.153ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cmpa
48762025-04-30T06:14:23.154ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "12677904-3412-406b-ab77-94c13290a715", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
48772025-04-30T06:14:23.154ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
48782025-04-30T06:14:23.155ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "725f5250-18dc-4650-91b5-3fd82385d143", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
48792025-04-30T06:14:23.155ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
48802025-04-30T06:14:23.156ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "70005bbb-45ec-43d8-a632-d4f092e5c9c2", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
48812025-04-30T06:14:23.156ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
48822025-04-30T06:14:23.158ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "61d66ced-5b62-4194-9c6f-9932386d2258", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
48832025-04-30T06:14:23.158ZTRACtest_basic: authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet
48842025-04-30T06:14:23.158ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
48852025-04-30T06:14:23.158ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
48862025-04-30T06:14:23.158ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
48872025-04-30T06:14:23.160ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
48882025-04-30T06:14:23.166ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48892025-04-30T06:14:23.171ZDEBGtest_basic: authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet result = Ok(())
48902025-04-30T06:14:23.171ZTRACtest_basic: authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet
48912025-04-30T06:14:23.171ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
48922025-04-30T06:14:23.171ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
48932025-04-30T06:14:23.171ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
48942025-04-30T06:14:23.172ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
48952025-04-30T06:14:23.175ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48962025-04-30T06:14:23.179ZDEBGtest_basic: authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet result = Ok(())
48972025-04-30T06:14:23.179ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
48982025-04-30T06:14:23.179ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
48992025-04-30T06:14:23.179ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
49002025-04-30T06:14:23.229ZDEBGtest_basic: begin collection from Sled Agent sled_agent_url = http://[::1]:42420
49012025-04-30T06:14:23.229ZDEBGtest_basic: client request SledAgent = http://[::1]:42420 body = None method = GET uri = http://[::1]:42420/inventory
49022025-04-30T06:14:23.230ZINFOtest_basic (dropshot): accepted connection local_addr = [::1]:42420 remote_addr = [::1]:62317 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
49032025-04-30T06:14:23.230ZTRACtest_basic (dropshot): incoming request local_addr = [::1]:42420 method = GET remote_addr = [::1]:62317 req_id = c0e23dc8-b6fb-46f5-8215-5981abdad6ef sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
49042025-04-30T06:14:23.231ZINFOtest_basic (dropshot): request completed latency_us = 738 local_addr = [::1]:42420 method = GET remote_addr = [::1]:62317 req_id = c0e23dc8-b6fb-46f5-8215-5981abdad6ef response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
49052025-04-30T06:14:23.231ZDEBGtest_basic: client response SledAgent = http://[::1]:42420 result = Ok(Response { url: "http://[::1]:42420/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0e23dc8-b6fb-46f5-8215-5981abdad6ef", "content-length": "2769", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
49062025-04-30T06:14:23.234ZDEBGtest_basic: begin collecting all keepers nkeeper_admin_clients = 0
49072025-04-30T06:14:23.234ZDEBGtest_basic: end collecting all keepers nkeeper_admin_clients = 0
49082025-04-30T06:14:23.234ZDEBGtest_basic: finished collection
49092025-04-30T06:14:23.237ZTRACtest_basic: authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory
49102025-04-30T06:14:23.237ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
49112025-04-30T06:14:23.237ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
49122025-04-30T06:14:23.237ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
49132025-04-30T06:14:23.238ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
49142025-04-30T06:14:23.242ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49152025-04-30T06:14:23.246ZDEBGtest_basic: authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory result = Ok(())
49162025-04-30T06:14:23.246ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
49172025-04-30T06:14:23.246ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
49182025-04-30T06:14:23.247ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
49192025-04-30T06:14:23.273ZWARNtest_basic (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
49202025-04-30T06:14:23.274ZWARNtest_basic: inventory collection failed error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
49212025-04-30T06:14:23.279ZTRACtest_basic: authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory
49222025-04-30T06:14:23.279ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
49232025-04-30T06:14:23.279ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
49242025-04-30T06:14:23.279ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
49252025-04-30T06:14:23.280ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
49262025-04-30T06:14:23.286ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49272025-04-30T06:14:23.291ZDEBGtest_basic: authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory result = Ok(())
49282025-04-30T06:14:23.292ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
49292025-04-30T06:14:23.292ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
49302025-04-30T06:14:23.292ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
49312025-04-30T06:14:23.297ZDEBGtest_basic: inventory_prune_one: nothing eligible for removal (too few) candidates = [(efcb0ee5-cafa-4692-9b75-fcee4732bdca, 0)]
49322025-04-30T06:14:23.297ZTRACtest_basic: lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
49332025-04-30T06:14:23.298ZTRACtest_basic (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 66 97 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 f............_mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:51574 req_id = 1a9c340d-12dc-4a47-b055-26200fdc3a53
49342025-04-30T06:14:23.298ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 26263,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51574 req_id = 1a9c340d-12dc-4a47-b055-26200fdc3a53
49352025-04-30T06:14:23.298ZDEBGtest_basic (store): query key key = _mgs._tcp
49362025-04-30T06:14:23.299ZDEBGtest_basic (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
49372025-04-30T06:14:23.299ZDEBGtest_basic (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:51574 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 37822, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 1a9c340d-12dc-4a47-b055-26200fdc3a53
49382025-04-30T06:14:23.300ZDEBGtest_basic: lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 37822, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2528, tv_nsec: 99188352 } })
49392025-04-30T06:14:23.301ZTRACtest_basic: lookup_service_targets: looking up SRV target
49402025-04-30T06:14:23.302ZTRACtest_basic (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 5a 22 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e Z"..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:58743 req_id = 37a0481b-91be-4eb4-bdff-ad0fc9cfebf8
49412025-04-30T06:14:23.303ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 23074,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:58743 req_id = 37a0481b-91be-4eb4-bdff-ad0fc9cfebf8
49422025-04-30T06:14:23.304ZDEBGtest_basic (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
49432025-04-30T06:14:23.304ZDEBGtest_basic (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:58743 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 37a0481b-91be-4eb4-bdff-ad0fc9cfebf8
49442025-04-30T06:14:23.356ZTRACtest_basic: lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
49452025-04-30T06:14:23.356ZDEBGtest_basic (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
49462025-04-30T06:14:23.356ZDEBGtest_basic (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
49472025-04-30T06:14:23.358ZTRACtest_basic (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: a1 99 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:63391 req_id = 719fe948-bd20-4a07-afcc-e7c493f14f64
49482025-04-30T06:14:23.359ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 41369,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63391 req_id = 719fe948-bd20-4a07-afcc-e7c493f14f64
49492025-04-30T06:14:23.359ZDEBGtest_basic (store): query key key = _clickhouse-admin-keeper._tcp
49502025-04-30T06:14:23.359ZERROtest_basic (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 41369, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:63391 req_id = 719fe948-bd20-4a07-afcc-e7c493f14f64
49512025-04-30T06:14:23.359ZTRACtest_basic (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
49522025-04-30T06:14:23.360ZDEBGtest_basic (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
49532025-04-30T06:14:23.360ZTRACtest_basic (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
49542025-04-30T06:14:23.360ZTRACtest_basic (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
49552025-04-30T06:14:23.361ZDEBGtest_basic: begin collection
49562025-04-30T06:14:23.361ZDEBGtest_basic: begin collection from MGS mgs_url = http://[::1]:37822
49572025-04-30T06:14:23.361ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/ignition
49582025-04-30T06:14:23.363ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "42ded280-6d06-49cc-95a4-6597dca343f6", "content-length": "835", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
49592025-04-30T06:14:23.363ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0
49602025-04-30T06:14:23.366ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c33de0aa-5963-45be-a0b0-ea9516f4ddd7", "content-length": "734", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
49612025-04-30T06:14:23.366ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=0
49622025-04-30T06:14:23.370ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0de9650c-0ca6-4dd4-985d-c1d4d14e8c78", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
49632025-04-30T06:14:23.371ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=1
49642025-04-30T06:14:23.374ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "47a09eaf-b4b5-4794-bddd-3b6c401dc597", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
49652025-04-30T06:14:23.374ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=0
49662025-04-30T06:14:23.379ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "abe8e87f-9583-47bf-a060-6db656655979", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
49672025-04-30T06:14:23.379ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=1
49682025-04-30T06:14:23.386ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c5c4cdfc-c651-4460-b678-f09b14129de7", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
49692025-04-30T06:14:23.386ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=0
49702025-04-30T06:14:23.390ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "90029290-2bc3-47d5-85c2-2bd591d7d532", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
49712025-04-30T06:14:23.391ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=1
49722025-04-30T06:14:23.395ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "85da188c-d178-4cc7-8f39-8a20e4ad721c", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
49732025-04-30T06:14:23.395ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cmpa
49742025-04-30T06:14:23.396ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "01259d00-0a23-4ef8-944b-30719f15382b", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
49752025-04-30T06:14:23.397ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
49762025-04-30T06:14:23.401ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d91f0ede-47f2-4c4e-885b-2d01f4c7abfd", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
49772025-04-30T06:14:23.402ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
49782025-04-30T06:14:23.403ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2cdc5d6b-ef46-4a16-beed-d0b93292f1f1", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
49792025-04-30T06:14:23.403ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
49802025-04-30T06:14:23.405ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "30c793df-aa69-4f7e-b4af-95b363c777e8", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
49812025-04-30T06:14:23.405ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1
49822025-04-30T06:14:23.408ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "053556e0-b6b8-4b9e-829f-aba45b39151f", "content-length": "734", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
49832025-04-30T06:14:23.408ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=0
49842025-04-30T06:14:23.412ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "76a498d6-b905-439a-9eab-09f60b4e9f11", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
49852025-04-30T06:14:23.412ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=1
49862025-04-30T06:14:23.417ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e22945c0-096f-4e76-b3fc-8e3c19d2df2f", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
49872025-04-30T06:14:23.418ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=0
49882025-04-30T06:14:23.423ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "22a35a3d-9f0b-49e7-9b36-38bfb32a3f6f", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
49892025-04-30T06:14:23.424ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=1
49902025-04-30T06:14:23.428ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "effc1340-eba7-4e5e-9e07-f8f960314271", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
49912025-04-30T06:14:23.428ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=0
49922025-04-30T06:14:23.430ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ebf925e2-4622-4fa7-ba80-6ee49b38da52", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
49932025-04-30T06:14:23.431ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=1
49942025-04-30T06:14:23.433ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "68c09c32-e960-4fad-bd0f-77bc098010cf", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
49952025-04-30T06:14:23.433ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cmpa
49962025-04-30T06:14:23.436ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e95d30ec-1450-4558-87ec-34796e6babe2", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
49972025-04-30T06:14:23.436ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
49982025-04-30T06:14:23.437ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b7d477bd-40fd-47dd-8144-4245a2c88044", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
49992025-04-30T06:14:23.437ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
50002025-04-30T06:14:23.439ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d478f77e-59b5-452e-81a8-9155f6d91fda", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
50012025-04-30T06:14:23.444ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
50022025-04-30T06:14:23.450ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "17d45e8f-f720-45ca-a934-87217f360ee6", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
50032025-04-30T06:14:23.450ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0
50042025-04-30T06:14:23.453ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bfd7c117-d418-4991-b6ac-5f9922aeab60", "content-length": "723", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
50052025-04-30T06:14:23.454ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=0
50062025-04-30T06:14:23.458ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6da3d6bd-3719-481d-9bec-6ceddb4550b5", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
50072025-04-30T06:14:23.458ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=1
50082025-04-30T06:14:23.463ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f0d885f1-0963-4b6c-a64a-bd95bcf11b2d", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
50092025-04-30T06:14:23.463ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=0
50102025-04-30T06:14:23.463ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9a4d1ccf-a506-4ae3-a49e-c4632a68f4ee", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
50112025-04-30T06:14:23.463ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=1
50122025-04-30T06:14:23.465ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a84fd033-40ff-45e1-96af-d30a30a0c26d", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
50132025-04-30T06:14:23.465ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=0
50142025-04-30T06:14:23.469ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d4878ab3-230d-4703-bd00-5883dbe5e0ff", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
50152025-04-30T06:14:23.469ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=1
50162025-04-30T06:14:23.474ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2720bcc7-d6ab-4e48-8145-aedc84581a6e", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
50172025-04-30T06:14:23.476ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cmpa
50182025-04-30T06:14:23.482ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2deb65b6-d926-4d1b-b52c-0a7b367f777b", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
50192025-04-30T06:14:23.482ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
50202025-04-30T06:14:23.484ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d6986704-e7bf-4540-b2a9-6698e8f4e0d2", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
50212025-04-30T06:14:23.484ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
50222025-04-30T06:14:23.485ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4d9c37c8-fe8b-462b-bb43-e6d11e2f6497", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
50232025-04-30T06:14:23.485ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
50242025-04-30T06:14:23.487ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "60c5450d-430f-47e4-b350-d89865d2ce69", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
50252025-04-30T06:14:23.488ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1
50262025-04-30T06:14:23.492ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6907e775-3aee-43e3-b4dc-2e269123e273", "content-length": "723", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
50272025-04-30T06:14:23.492ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=0
50282025-04-30T06:14:23.500ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "09b9579e-4c8d-4509-b7a3-71d815778561", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
50292025-04-30T06:14:23.500ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=1
50302025-04-30T06:14:23.588ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e2890460-c308-46ce-9c81-f015af6ba7a8", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
50312025-04-30T06:14:23.589ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=0
50322025-04-30T06:14:23.597ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "693c8eca-c294-4a93-850d-ab1abb36d106", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
50332025-04-30T06:14:23.598ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=1
50342025-04-30T06:14:23.606ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cbab5af5-c866-440a-a2f3-8e1cb318e135", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
50352025-04-30T06:14:23.607ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=0
50362025-04-30T06:14:23.612ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5eb5d1df-ca8e-474e-b6ce-f9efe183983a", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
50372025-04-30T06:14:23.613ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=1
50382025-04-30T06:14:23.639ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0d115dbc-f71b-4170-8ef4-b08501412c42", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
50392025-04-30T06:14:23.646ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cmpa
50402025-04-30T06:14:23.647ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a1af8228-3404-4cb2-aa60-f670b17ffbd9", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
50412025-04-30T06:14:23.648ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
50422025-04-30T06:14:23.649ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "718d4425-b9c5-4da8-92f2-118c3739bd16", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
50432025-04-30T06:14:23.649ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
50442025-04-30T06:14:23.650ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9f81f155-2951-4290-8d1d-114bf0f75f2a", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
50452025-04-30T06:14:23.650ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
50462025-04-30T06:14:23.651ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "dba8ee89-826d-4637-afd0-24951e081392", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
50472025-04-30T06:14:23.651ZTRACtest_basic: authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet
50482025-04-30T06:14:23.651ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
50492025-04-30T06:14:23.651ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
50502025-04-30T06:14:23.651ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
50512025-04-30T06:14:23.652ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
50522025-04-30T06:14:23.655ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50532025-04-30T06:14:23.663ZDEBGtest_basic: authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet result = Ok(())
50542025-04-30T06:14:23.663ZTRACtest_basic: authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet
50552025-04-30T06:14:23.663ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
50562025-04-30T06:14:23.664ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
50572025-04-30T06:14:23.664ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
50582025-04-30T06:14:23.665ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
50592025-04-30T06:14:23.686ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50602025-04-30T06:14:23.695ZDEBGtest_basic: authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet result = Ok(())
50612025-04-30T06:14:23.695ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
50622025-04-30T06:14:23.695ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
50632025-04-30T06:14:23.696ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
50642025-04-30T06:14:23.830ZDEBGtest_basic: begin collection from Sled Agent sled_agent_url = http://[::1]:42420
50652025-04-30T06:14:23.830ZDEBGtest_basic: client request SledAgent = http://[::1]:42420 body = None method = GET uri = http://[::1]:42420/inventory
50662025-04-30T06:14:23.830ZINFOtest_basic (dropshot): accepted connection local_addr = [::1]:42420 remote_addr = [::1]:55075 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
50672025-04-30T06:14:23.830ZDEBGtest_basic (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
50682025-04-30T06:14:23.830ZDEBGtest_basic (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
50692025-04-30T06:14:23.831ZDEBGtest_basic (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812
50702025-04-30T06:14:23.832ZTRACtest_basic (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
50712025-04-30T06:14:23.833ZTRACtest_basic (dropshot): incoming request local_addr = [::1]:42420 method = GET remote_addr = [::1]:55075 req_id = 73d19564-2827-49cd-a3ed-504c22ab2428 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
50722025-04-30T06:14:23.833ZINFOtest_basic (dropshot): request completed latency_us = 521 local_addr = [::1]:42420 method = GET remote_addr = [::1]:55075 req_id = 73d19564-2827-49cd-a3ed-504c22ab2428 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
50732025-04-30T06:14:23.834ZDEBGtest_basic: client response SledAgent = http://[::1]:42420 result = Ok(Response { url: "http://[::1]:42420/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "73d19564-2827-49cd-a3ed-504c22ab2428", "content-length": "2769", "date": "Wed, 30 Apr 2025 06:14:23 GMT"} })
50742025-04-30T06:14:23.836ZDEBGtest_basic: begin collecting all keepers nkeeper_admin_clients = 0
50752025-04-30T06:14:23.836ZDEBGtest_basic: end collecting all keepers nkeeper_admin_clients = 0
50762025-04-30T06:14:23.836ZDEBGtest_basic: finished collection
50772025-04-30T06:14:23.845ZTRACtest_basic: authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory
50782025-04-30T06:14:23.845ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
50792025-04-30T06:14:23.845ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
50802025-04-30T06:14:23.845ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
50812025-04-30T06:14:23.849ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
50822025-04-30T06:14:23.849ZDEBGtest_basic (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_rows = 2 table_name = fields_string
50832025-04-30T06:14:23.849ZTRACtest_basic (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
50842025-04-30T06:14:23.854ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50852025-04-30T06:14:23.867ZDEBGtest_basic: authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory result = Ok(())
50862025-04-30T06:14:23.867ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
50872025-04-30T06:14:23.867ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
50882025-04-30T06:14:23.868ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
50892025-04-30T06:14:23.869ZDEBGtest_basic (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_rows = 1 table_name = measurements_i64
50902025-04-30T06:14:23.869ZTRACtest_basic (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
50912025-04-30T06:14:23.909ZWARNtest_basic (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
50922025-04-30T06:14:23.909ZWARNtest_basic: inventory collection failed error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
50932025-04-30T06:14:23.930ZTRACtest_basic: authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory
50942025-04-30T06:14:23.930ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
50952025-04-30T06:14:23.930ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
50962025-04-30T06:14:23.931ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
50972025-04-30T06:14:23.932ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
50982025-04-30T06:14:23.932ZWARNtest_basic (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
50992025-04-30T06:14:23.936ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51002025-04-30T06:14:23.941ZDEBGtest_basic: authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory result = Ok(())
51012025-04-30T06:14:23.941ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
51022025-04-30T06:14:23.941ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
51032025-04-30T06:14:23.941ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
51042025-04-30T06:14:23.947ZDEBGtest_basic: inventory_prune_one: nothing eligible for removal (too few) candidates = [(efcb0ee5-cafa-4692-9b75-fcee4732bdca, 0)]
51052025-04-30T06:14:23.948ZTRACtest_basic: lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
51062025-04-30T06:14:23.949ZTRACtest_basic (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: da 07 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:62050 req_id = 677b264b-7723-434b-b049-bcc20c808c05
51072025-04-30T06:14:23.950ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 55815,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62050 req_id = 677b264b-7723-434b-b049-bcc20c808c05
51082025-04-30T06:14:23.950ZDEBGtest_basic (store): query key key = _mgs._tcp
51092025-04-30T06:14:23.950ZDEBGtest_basic (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
51102025-04-30T06:14:23.951ZDEBGtest_basic (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:62050 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 37822, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 677b264b-7723-434b-b049-bcc20c808c05
51112025-04-30T06:14:23.951ZDEBGtest_basic: lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 37822, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2528, tv_nsec: 750679395 } })
51122025-04-30T06:14:23.951ZTRACtest_basic: lookup_service_targets: looking up SRV target
51132025-04-30T06:14:23.952ZTRACtest_basic (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: f9 59 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .Y..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:62309 req_id = d6ecee15-d4cb-4e75-8c17-904ac3534073
51142025-04-30T06:14:23.952ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 63833,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62309 req_id = d6ecee15-d4cb-4e75-8c17-904ac3534073
51152025-04-30T06:14:23.952ZDEBGtest_basic (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
51162025-04-30T06:14:23.952ZDEBGtest_basic (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:62309 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = d6ecee15-d4cb-4e75-8c17-904ac3534073
51172025-04-30T06:14:24.104ZTRACtest_basic: lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
51182025-04-30T06:14:24.108ZTRACtest_basic (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 7c 78 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c |x..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:63010 req_id = 57028a73-a950-4d57-b07f-8e0f184691cd
51192025-04-30T06:14:24.108ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 31864,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63010 req_id = 57028a73-a950-4d57-b07f-8e0f184691cd
51202025-04-30T06:14:24.108ZDEBGtest_basic (store): query key key = _clickhouse-admin-keeper._tcp
51212025-04-30T06:14:24.109ZERROtest_basic (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 31864, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:63010 req_id = 57028a73-a950-4d57-b07f-8e0f184691cd
51222025-04-30T06:14:24.109ZDEBGtest_basic: begin collection
51232025-04-30T06:14:24.109ZDEBGtest_basic: begin collection from MGS mgs_url = http://[::1]:37822
51242025-04-30T06:14:24.109ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/ignition
51252025-04-30T06:14:24.140ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "5762f14d-5cd1-40bc-bcb9-d956e60a98da", "content-length": "835", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
51262025-04-30T06:14:24.141ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0
51272025-04-30T06:14:24.158ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "459277eb-0c98-4b5e-ae79-1fa775433344", "content-length": "734", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
51282025-04-30T06:14:24.159ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=0
51292025-04-30T06:14:24.161ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8184dd15-8afe-4929-af75-23b34f2ab841", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
51302025-04-30T06:14:24.161ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=1
51312025-04-30T06:14:24.163ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "32064aa1-e7a7-4e34-8633-678845bf5b45", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
51322025-04-30T06:14:24.163ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=0
51332025-04-30T06:14:24.165ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d9ee9fdc-4522-44c3-bd16-9c4539414585", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
51342025-04-30T06:14:24.165ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=1
51352025-04-30T06:14:24.166ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "62654083-ad3b-4d9f-931b-a553bce9b524", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
51362025-04-30T06:14:24.166ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=0
51372025-04-30T06:14:24.169ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d30a33a8-41d2-4c40-a5aa-2ca0b3e700cc", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
51382025-04-30T06:14:24.169ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=1
51392025-04-30T06:14:24.172ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "525f9ffe-56f2-4d73-bc96-98b62b626fa0", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
51402025-04-30T06:14:24.172ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cmpa
51412025-04-30T06:14:24.174ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9ab4028e-be0f-4a3a-9f3b-3f51139f3854", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
51422025-04-30T06:14:24.174ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
51432025-04-30T06:14:24.175ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6525d7a6-171f-4368-bb90-5b44200a0bf9", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
51442025-04-30T06:14:24.175ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
51452025-04-30T06:14:24.176ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1130a4c1-f02a-44ec-bac5-91b75ec52338", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
51462025-04-30T06:14:24.176ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
51472025-04-30T06:14:24.177ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "822dfd89-94de-4bbc-ae0f-3c3dae709e2d", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
51482025-04-30T06:14:24.177ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1
51492025-04-30T06:14:24.179ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "35a6d6b1-48e4-4444-b0b8-754aa6e70085", "content-length": "734", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
51502025-04-30T06:14:24.179ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=0
51512025-04-30T06:14:24.184ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b897c1ff-9958-4475-86bd-1667d05a3836", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
51522025-04-30T06:14:24.185ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=1
51532025-04-30T06:14:24.188ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "01c85e74-3676-4fe9-bdba-0c5b5518ebf9", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
51542025-04-30T06:14:24.189ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=0
51552025-04-30T06:14:24.191ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "32d40502-6c02-431a-b54e-b26bd551f317", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
51562025-04-30T06:14:24.191ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=1
51572025-04-30T06:14:24.199ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3b8acdf3-523b-4a02-9ed0-0c26e73c03ee", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
51582025-04-30T06:14:24.199ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=0
51592025-04-30T06:14:24.202ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b9619407-770c-4466-a48b-2ea2263f87d0", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
51602025-04-30T06:14:24.202ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=1
51612025-04-30T06:14:24.205ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d464b5be-af88-41d7-bddb-eaab09398f2f", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
51622025-04-30T06:14:24.206ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cmpa
51632025-04-30T06:14:24.211ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "811fceb4-f2cd-40c4-ba6b-5a683bc2eae5", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
51642025-04-30T06:14:24.211ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
51652025-04-30T06:14:24.217ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "daac1e2f-abdb-4828-a24f-934a946c4779", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
51662025-04-30T06:14:24.217ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
51672025-04-30T06:14:24.219ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "889d02bc-e621-4f54-9545-342637f4d62c", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
51682025-04-30T06:14:24.220ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
51692025-04-30T06:14:24.222ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5c4cc91b-8877-4056-9f01-d2d22aea755f", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
51702025-04-30T06:14:24.222ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0
51712025-04-30T06:14:24.225ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ac738a53-6ec4-4b67-93eb-406698ae2c3a", "content-length": "723", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
51722025-04-30T06:14:24.225ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=0
51732025-04-30T06:14:24.233ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e27410c2-84ea-4aac-99e2-d42a85336626", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
51742025-04-30T06:14:24.233ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=1
51752025-04-30T06:14:24.237ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7065b7be-fc4f-4384-9c33-396415340f32", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
51762025-04-30T06:14:24.238ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=0
51772025-04-30T06:14:24.241ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "415a4887-f473-4264-b0ca-d70d41ac82f4", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
51782025-04-30T06:14:24.242ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=1
51792025-04-30T06:14:24.245ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "304034d3-1ae0-45d5-8c61-e84588c1e51b", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
51802025-04-30T06:14:24.245ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=0
51812025-04-30T06:14:24.249ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e34cbf2f-a7a3-495e-970e-8236a3dde4ee", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
51822025-04-30T06:14:24.249ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=1
51832025-04-30T06:14:24.254ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1abeb6a9-8acd-4377-9504-64257a1962e4", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
51842025-04-30T06:14:24.255ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cmpa
51852025-04-30T06:14:24.256ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2fc0b421-7ab6-4f19-bd79-c1e378f769fd", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
51862025-04-30T06:14:24.256ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
51872025-04-30T06:14:24.259ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0080c91b-3ca6-44da-98bf-edebb25b3444", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
51882025-04-30T06:14:24.259ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
51892025-04-30T06:14:24.260ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f3323f4c-a975-4b42-81e2-b3a94712bce7", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
51902025-04-30T06:14:24.261ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
51912025-04-30T06:14:24.266ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1f452c66-95e0-4fd5-ac68-bd49970a59e0", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
51922025-04-30T06:14:24.266ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1
51932025-04-30T06:14:24.270ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b4b07908-987c-44b2-ad36-8a64a4d79e11", "content-length": "723", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
51942025-04-30T06:14:24.279ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=0
51952025-04-30T06:14:24.280ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f95c3ce-b46e-4a58-94a8-1bccb25cce3b", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
51962025-04-30T06:14:24.280ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=1
51972025-04-30T06:14:24.285ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa42ea14-3835-4d57-836a-173b82123e19", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
51982025-04-30T06:14:24.286ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=0
51992025-04-30T06:14:24.290ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa2767dc-c597-4043-8ea8-0fd1576bd9bc", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
52002025-04-30T06:14:24.292ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=1
52012025-04-30T06:14:24.296ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "976c33dd-b0ef-48f0-ba8d-91aed64e3075", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
52022025-04-30T06:14:24.296ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=0
52032025-04-30T06:14:24.298ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "631bd164-04f4-468a-b959-cc7f6b3aa640", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
52042025-04-30T06:14:24.298ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=1
52052025-04-30T06:14:24.302ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "679ee864-42af-482e-a088-b64f15625379", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
52062025-04-30T06:14:24.302ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cmpa
52072025-04-30T06:14:24.303ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "762e7600-5c6b-4dfb-8aca-ed2a5541f33e", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
52082025-04-30T06:14:24.303ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
52092025-04-30T06:14:24.313ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "70ad095a-39dc-44bf-88cd-848e125e6212", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
52102025-04-30T06:14:24.313ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
52112025-04-30T06:14:24.315ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "27105ee9-a6f9-4d6b-80bb-a891723b19bd", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
52122025-04-30T06:14:24.315ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
52132025-04-30T06:14:24.317ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a4abda6-688c-4ea9-bb6d-84ba3bfa4704", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
52142025-04-30T06:14:24.317ZTRACtest_basic: authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet
52152025-04-30T06:14:24.317ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
52162025-04-30T06:14:24.317ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
52172025-04-30T06:14:24.317ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
52182025-04-30T06:14:24.323ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
52192025-04-30T06:14:24.328ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52202025-04-30T06:14:24.333ZDEBGtest_basic: authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet result = Ok(())
52212025-04-30T06:14:24.333ZTRACtest_basic: authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet
52222025-04-30T06:14:24.333ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
52232025-04-30T06:14:24.333ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
52242025-04-30T06:14:24.333ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
52252025-04-30T06:14:24.334ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
52262025-04-30T06:14:24.338ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52272025-04-30T06:14:24.342ZDEBGtest_basic: authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet result = Ok(())
52282025-04-30T06:14:24.342ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
52292025-04-30T06:14:24.342ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
52302025-04-30T06:14:24.343ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
52312025-04-30T06:14:24.402ZDEBGtest_basic: begin collection from Sled Agent sled_agent_url = http://[::1]:42420
52322025-04-30T06:14:24.402ZDEBGtest_basic: client request SledAgent = http://[::1]:42420 body = None method = GET uri = http://[::1]:42420/inventory
52332025-04-30T06:14:24.404ZINFOtest_basic (dropshot): accepted connection local_addr = [::1]:42420 remote_addr = [::1]:58182 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
52342025-04-30T06:14:24.404ZDEBGtest_basic (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
52352025-04-30T06:14:24.404ZDEBGtest_basic (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
52362025-04-30T06:14:24.415ZTRACtest_basic (dropshot): incoming request local_addr = [::1]:42420 method = GET remote_addr = [::1]:58182 req_id = d6b2c5d9-d58d-486a-a0cf-c36f025b44ad sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
52372025-04-30T06:14:24.416ZTRACtest_basic (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
52382025-04-30T06:14:24.419ZDEBGtest_basic (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
52392025-04-30T06:14:24.419ZTRACtest_basic (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
52402025-04-30T06:14:24.419ZTRACtest_basic (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
52412025-04-30T06:14:24.420ZINFOtest_basic (dropshot): request completed latency_us = 4651 local_addr = [::1]:42420 method = GET remote_addr = [::1]:58182 req_id = d6b2c5d9-d58d-486a-a0cf-c36f025b44ad response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
52422025-04-30T06:14:24.421ZDEBGtest_basic: client response SledAgent = http://[::1]:42420 result = Ok(Response { url: "http://[::1]:42420/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "d6b2c5d9-d58d-486a-a0cf-c36f025b44ad", "content-length": "2769", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
52432025-04-30T06:14:24.432ZDEBGtest_basic: begin collecting all keepers nkeeper_admin_clients = 0
52442025-04-30T06:14:24.432ZDEBGtest_basic: end collecting all keepers nkeeper_admin_clients = 0
52452025-04-30T06:14:24.432ZDEBGtest_basic: finished collection
52462025-04-30T06:14:24.437ZTRACtest_basic: authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory
52472025-04-30T06:14:24.437ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
52482025-04-30T06:14:24.437ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
52492025-04-30T06:14:24.437ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
52502025-04-30T06:14:24.441ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
52512025-04-30T06:14:24.446ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52522025-04-30T06:14:24.456ZDEBGtest_basic: authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory result = Ok(())
52532025-04-30T06:14:24.457ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
52542025-04-30T06:14:24.457ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
52552025-04-30T06:14:24.459ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
52562025-04-30T06:14:24.502ZWARNtest_basic (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
52572025-04-30T06:14:24.502ZWARNtest_basic: inventory collection failed error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
52582025-04-30T06:14:24.508ZTRACtest_basic: authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory
52592025-04-30T06:14:24.508ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
52602025-04-30T06:14:24.508ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
52612025-04-30T06:14:24.508ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
52622025-04-30T06:14:24.511ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
52632025-04-30T06:14:24.515ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52642025-04-30T06:14:24.519ZDEBGtest_basic: authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory result = Ok(())
52652025-04-30T06:14:24.519ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
52662025-04-30T06:14:24.519ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
52672025-04-30T06:14:24.520ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
52682025-04-30T06:14:24.524ZDEBGtest_basic: inventory_prune_one: nothing eligible for removal (too few) candidates = [(efcb0ee5-cafa-4692-9b75-fcee4732bdca, 0)]
52692025-04-30T06:14:24.524ZTRACtest_basic: lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
52702025-04-30T06:14:24.525ZTRACtest_basic (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 97 4d 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .M..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:56510 req_id = 9cf65974-7374-4b58-af89-ba83a1fa0166
52712025-04-30T06:14:24.525ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 38733,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:56510 req_id = 9cf65974-7374-4b58-af89-ba83a1fa0166
52722025-04-30T06:14:24.525ZDEBGtest_basic (store): query key key = _mgs._tcp
52732025-04-30T06:14:24.525ZDEBGtest_basic (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
52742025-04-30T06:14:24.525ZDEBGtest_basic (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:56510 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 37822, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 9cf65974-7374-4b58-af89-ba83a1fa0166
52752025-04-30T06:14:24.526ZDEBGtest_basic: lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 37822, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2529, tv_nsec: 324999892 } })
52762025-04-30T06:14:24.526ZTRACtest_basic: lookup_service_targets: looking up SRV target
52772025-04-30T06:14:24.526ZTRACtest_basic (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 59 15 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e Y...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:61797 req_id = 943017fa-9a95-41cb-8ae2-a1853c49a410
52782025-04-30T06:14:24.526ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 22805,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:61797 req_id = 943017fa-9a95-41cb-8ae2-a1853c49a410
52792025-04-30T06:14:24.526ZDEBGtest_basic (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
52802025-04-30T06:14:24.526ZDEBGtest_basic (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:61797 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 943017fa-9a95-41cb-8ae2-a1853c49a410
52812025-04-30T06:14:24.576ZTRACtest_basic: lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
52822025-04-30T06:14:24.576ZTRACtest_basic (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 9e c4 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:64493 req_id = b5404e78-6218-4fb1-a1e7-5b9cf34b8f4f
52832025-04-30T06:14:24.577ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 40644,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64493 req_id = b5404e78-6218-4fb1-a1e7-5b9cf34b8f4f
52842025-04-30T06:14:24.577ZDEBGtest_basic (store): query key key = _clickhouse-admin-keeper._tcp
52852025-04-30T06:14:24.577ZERROtest_basic (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 40644, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:64493 req_id = b5404e78-6218-4fb1-a1e7-5b9cf34b8f4f
52862025-04-30T06:14:24.577ZDEBGtest_basic: begin collection
52872025-04-30T06:14:24.577ZDEBGtest_basic: begin collection from MGS mgs_url = http://[::1]:37822
52882025-04-30T06:14:24.577ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/ignition
52892025-04-30T06:14:24.579ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "f971bb0d-cbd6-4ef6-96db-ee4ac0700278", "content-length": "835", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
52902025-04-30T06:14:24.579ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0
52912025-04-30T06:14:24.580ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "97c35a6f-c4c1-428c-9d6d-e9a3675243a9", "content-length": "734", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
52922025-04-30T06:14:24.581ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=0
52932025-04-30T06:14:24.583ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5c6143bb-dde7-4ef8-ac1f-8b6a1258330c", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
52942025-04-30T06:14:24.583ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=1
52952025-04-30T06:14:24.585ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1219219d-8a55-4139-b341-79b7e3b343c3", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
52962025-04-30T06:14:24.585ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=0
52972025-04-30T06:14:24.587ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "281a63e1-63df-4a35-b61d-cb71175619a3", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
52982025-04-30T06:14:24.587ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=1
52992025-04-30T06:14:24.590ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa597524-92a2-4546-8bac-c6bad432ceae", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
53002025-04-30T06:14:24.590ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=0
53012025-04-30T06:14:24.592ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e4a91118-963f-4e65-8e3d-1f75a8e9eadf", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
53022025-04-30T06:14:24.592ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=1
53032025-04-30T06:14:24.593ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0e6ae570-8a6a-41ed-8d82-5857867e7739", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
53042025-04-30T06:14:24.594ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cmpa
53052025-04-30T06:14:24.595ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0b6d4ae2-a284-4f44-a422-14cbe37802b9", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
53062025-04-30T06:14:24.595ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
53072025-04-30T06:14:24.596ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "574dc6fc-4e86-4e77-b877-ae4ceee399fd", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
53082025-04-30T06:14:24.596ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
53092025-04-30T06:14:24.597ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "dae693e6-05af-447c-9ca0-9a3703f81c02", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
53102025-04-30T06:14:24.597ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
53112025-04-30T06:14:24.600ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0579f5b4-4cfe-4e6e-acfb-fc24f7230f99", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
53122025-04-30T06:14:24.601ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1
53132025-04-30T06:14:24.603ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "29c36ef0-f1aa-4d12-8b2c-6701cb4dc039", "content-length": "734", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
53142025-04-30T06:14:24.603ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=0
53152025-04-30T06:14:24.606ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "32e0c394-760a-4c63-9635-9e66f0d25252", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
53162025-04-30T06:14:24.606ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=1
53172025-04-30T06:14:24.607ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f912f94d-822f-4036-8af4-80d217b8688c", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
53182025-04-30T06:14:24.608ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=0
53192025-04-30T06:14:24.609ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e0dd0cca-d4f9-4d98-8d4f-0a44c4749580", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
53202025-04-30T06:14:24.609ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=1
53212025-04-30T06:14:24.611ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d768e197-a816-4764-8b72-704ad67471eb", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
53222025-04-30T06:14:24.611ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=0
53232025-04-30T06:14:24.613ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3ad931f0-39c0-4d6b-bd79-c898db77db9e", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
53242025-04-30T06:14:24.613ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=1
53252025-04-30T06:14:24.615ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a4e0d49e-4e24-4239-a731-3fe8d9f3ee01", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
53262025-04-30T06:14:24.615ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cmpa
53272025-04-30T06:14:24.616ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ae2ac1b6-f787-45bb-8934-9294af6359ae", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
53282025-04-30T06:14:24.616ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
53292025-04-30T06:14:24.617ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ff070383-dd3b-4569-843d-cf4111281167", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
53302025-04-30T06:14:24.617ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
53312025-04-30T06:14:24.618ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e83dc3ed-d7fb-4a63-9c78-94a705d18339", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
53322025-04-30T06:14:24.618ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
53332025-04-30T06:14:24.619ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a1a340a4-dd32-4d9b-8d57-eee9f11caaa0", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
53342025-04-30T06:14:24.619ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0
53352025-04-30T06:14:24.620ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b4649259-118d-4f18-b6e0-1e36048ef023", "content-length": "723", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
53362025-04-30T06:14:24.620ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=0
53372025-04-30T06:14:24.622ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d6f03b12-a856-4dbf-b020-fe0ba2d11b29", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
53382025-04-30T06:14:24.622ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=1
53392025-04-30T06:14:24.623ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5d3aa35d-5737-49eb-9d6e-9d8086a146af", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
53402025-04-30T06:14:24.623ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=0
53412025-04-30T06:14:24.625ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c05a12a0-234e-4d9c-9d9a-a8e6fa1dfe19", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
53422025-04-30T06:14:24.625ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=1
53432025-04-30T06:14:24.627ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "73edad8b-dfc3-4b1b-b465-8e5e3fa25a05", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
53442025-04-30T06:14:24.627ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=0
53452025-04-30T06:14:24.629ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "616a858b-5388-40b0-a77f-4007aad0ed9e", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
53462025-04-30T06:14:24.629ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=1
53472025-04-30T06:14:24.630ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "10812993-4e41-4744-90c7-784fc11db5b5", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
53482025-04-30T06:14:24.631ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cmpa
53492025-04-30T06:14:24.631ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c73dbe17-89ba-445c-98cd-3b0af22cfb6b", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
53502025-04-30T06:14:24.631ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
53512025-04-30T06:14:24.632ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "222af978-f969-472b-8347-32daade6dc9e", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
53522025-04-30T06:14:24.632ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
53532025-04-30T06:14:24.633ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4b3a6eea-91e5-4859-96d2-adb747d2cefe", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
53542025-04-30T06:14:24.634ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
53552025-04-30T06:14:24.634ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e2e7f7e0-1214-46fc-877a-9a210ba36bd8", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
53562025-04-30T06:14:24.635ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1
53572025-04-30T06:14:24.636ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9fac1b7f-d825-4992-9952-8c5d7eceb384", "content-length": "723", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
53582025-04-30T06:14:24.636ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=0
53592025-04-30T06:14:24.638ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c5ab2901-99c8-4104-a9f2-5ee4a37a0581", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
53602025-04-30T06:14:24.639ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=1
53612025-04-30T06:14:24.641ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "26fdc6f1-3d5d-473f-b1dc-0e41c924b02c", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
53622025-04-30T06:14:24.641ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=0
53632025-04-30T06:14:24.645ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "473c0b34-e72f-4def-890c-c018bfa1af04", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
53642025-04-30T06:14:24.645ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=1
53652025-04-30T06:14:24.648ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "98852a49-813d-4a9f-9e45-4229f96b8db5", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
53662025-04-30T06:14:24.648ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=0
53672025-04-30T06:14:24.651ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "eacdcbe2-526b-446e-a0d5-a3e1fbb05dfc", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
53682025-04-30T06:14:24.651ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=1
53692025-04-30T06:14:24.655ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "918ca545-11ed-4f57-b92d-46eb8249ea44", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
53702025-04-30T06:14:24.655ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cmpa
53712025-04-30T06:14:24.661ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "af78df14-e2c0-4cf4-aaa2-c06f6012b32a", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
53722025-04-30T06:14:24.661ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
53732025-04-30T06:14:24.664ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8d47745f-b59a-4057-9253-6663992d599d", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
53742025-04-30T06:14:24.664ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
53752025-04-30T06:14:24.664ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6ab74442-f529-4120-972f-68468ef916cc", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
53762025-04-30T06:14:24.664ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
53772025-04-30T06:14:24.665ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a1678ddd-a4ac-46e9-944f-166ce3b7fd77", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
53782025-04-30T06:14:24.665ZTRACtest_basic: authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet
53792025-04-30T06:14:24.665ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
53802025-04-30T06:14:24.665ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
53812025-04-30T06:14:24.665ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
53822025-04-30T06:14:24.669ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
53832025-04-30T06:14:24.674ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53842025-04-30T06:14:24.680ZDEBGtest_basic: authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet result = Ok(())
53852025-04-30T06:14:24.680ZTRACtest_basic: authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet
53862025-04-30T06:14:24.680ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
53872025-04-30T06:14:24.680ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
53882025-04-30T06:14:24.681ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
53892025-04-30T06:14:24.681ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
53902025-04-30T06:14:24.686ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53912025-04-30T06:14:24.694ZDEBGtest_basic: authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet result = Ok(())
53922025-04-30T06:14:24.695ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
53932025-04-30T06:14:24.695ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
53942025-04-30T06:14:24.697ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
53952025-04-30T06:14:24.813ZDEBGtest_basic: begin collection from Sled Agent sled_agent_url = http://[::1]:42420
53962025-04-30T06:14:24.813ZDEBGtest_basic: client request SledAgent = http://[::1]:42420 body = None method = GET uri = http://[::1]:42420/inventory
53972025-04-30T06:14:24.814ZINFOtest_basic (dropshot): accepted connection local_addr = [::1]:42420 remote_addr = [::1]:57232 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
53982025-04-30T06:14:24.814ZDEBGtest_basic (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
53992025-04-30T06:14:24.814ZDEBGtest_basic (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
54002025-04-30T06:14:24.814ZDEBGtest_basic (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812
54012025-04-30T06:14:24.815ZTRACtest_basic (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
54022025-04-30T06:14:24.815ZTRACtest_basic (dropshot): incoming request local_addr = [::1]:42420 method = GET remote_addr = [::1]:57232 req_id = fbc9e25a-6bc3-42d7-83c0-6b0c05445fbe sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
54032025-04-30T06:14:24.816ZINFOtest_basic (dropshot): request completed latency_us = 440 local_addr = [::1]:42420 method = GET remote_addr = [::1]:57232 req_id = fbc9e25a-6bc3-42d7-83c0-6b0c05445fbe response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
54042025-04-30T06:14:24.816ZDEBGtest_basic: client response SledAgent = http://[::1]:42420 result = Ok(Response { url: "http://[::1]:42420/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "fbc9e25a-6bc3-42d7-83c0-6b0c05445fbe", "content-length": "2769", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
54052025-04-30T06:14:24.818ZDEBGtest_basic: begin collecting all keepers nkeeper_admin_clients = 0
54062025-04-30T06:14:24.818ZDEBGtest_basic: end collecting all keepers nkeeper_admin_clients = 0
54072025-04-30T06:14:24.818ZDEBGtest_basic: finished collection
54082025-04-30T06:14:24.822ZTRACtest_basic: authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory
54092025-04-30T06:14:24.822ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
54102025-04-30T06:14:24.822ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
54112025-04-30T06:14:24.822ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
54122025-04-30T06:14:24.823ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
54132025-04-30T06:14:24.833ZDEBGtest_basic (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_rows = 2 table_name = fields_string
54142025-04-30T06:14:24.833ZTRACtest_basic (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
54152025-04-30T06:14:24.835ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54162025-04-30T06:14:24.842ZDEBGtest_basic: authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory result = Ok(())
54172025-04-30T06:14:24.842ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
54182025-04-30T06:14:24.842ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
54192025-04-30T06:14:24.843ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
54202025-04-30T06:14:24.843ZDEBGtest_basic (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_rows = 1 table_name = measurements_i64
54212025-04-30T06:14:24.843ZTRACtest_basic (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
54222025-04-30T06:14:24.871ZWARNtest_basic (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
54232025-04-30T06:14:24.871ZWARNtest_basic: inventory collection failed error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
54242025-04-30T06:14:24.875ZTRACtest_basic: authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory
54252025-04-30T06:14:24.875ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
54262025-04-30T06:14:24.875ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
54272025-04-30T06:14:24.875ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
54282025-04-30T06:14:24.876ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
54292025-04-30T06:14:24.879ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54302025-04-30T06:14:24.884ZDEBGtest_basic: authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory result = Ok(())
54312025-04-30T06:14:24.884ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
54322025-04-30T06:14:24.884ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
54332025-04-30T06:14:24.885ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
54342025-04-30T06:14:24.888ZDEBGtest_basic: inventory_prune_one: nothing eligible for removal (too few) candidates = [(efcb0ee5-cafa-4692-9b75-fcee4732bdca, 0)]
54352025-04-30T06:14:24.888ZTRACtest_basic: lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
54362025-04-30T06:14:24.890ZTRACtest_basic (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: a7 1b 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:49664 req_id = 0b3352ce-7b54-46d6-8cb3-1d1ec76c3559
54372025-04-30T06:14:24.890ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 42779,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:49664 req_id = 0b3352ce-7b54-46d6-8cb3-1d1ec76c3559
54382025-04-30T06:14:24.890ZDEBGtest_basic (store): query key key = _mgs._tcp
54392025-04-30T06:14:24.891ZDEBGtest_basic (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
54402025-04-30T06:14:24.891ZDEBGtest_basic (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:49664 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 37822, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 0b3352ce-7b54-46d6-8cb3-1d1ec76c3559
54412025-04-30T06:14:24.892ZDEBGtest_basic: lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 37822, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2529, tv_nsec: 691329699 } })
54422025-04-30T06:14:24.892ZTRACtest_basic: lookup_service_targets: looking up SRV target
54432025-04-30T06:14:24.892ZTRACtest_basic (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: f5 14 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:63978 req_id = b58b0edc-f549-40e8-a0b8-cf63154d276d
54442025-04-30T06:14:24.892ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 62740,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63978 req_id = b58b0edc-f549-40e8-a0b8-cf63154d276d
54452025-04-30T06:14:24.896ZDEBGtest_basic (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
54462025-04-30T06:14:24.897ZDEBGtest_basic (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:63978 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = b58b0edc-f549-40e8-a0b8-cf63154d276d
54472025-04-30T06:14:24.966ZTRACtest_basic: lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
54482025-04-30T06:14:24.967ZWARNtest_basic (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
54492025-04-30T06:14:24.967ZTRACtest_basic (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 7e cd 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ~............_cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:49176 req_id = 54429a39-9f37-4012-b121-09eb1b6d2bd6
54502025-04-30T06:14:24.968ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 32461,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:49176 req_id = 54429a39-9f37-4012-b121-09eb1b6d2bd6
54512025-04-30T06:14:24.969ZDEBGtest_basic (store): query key key = _clickhouse-admin-keeper._tcp
54522025-04-30T06:14:24.969ZERROtest_basic (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 32461, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:49176 req_id = 54429a39-9f37-4012-b121-09eb1b6d2bd6
54532025-04-30T06:14:24.969ZDEBGtest_basic: begin collection
54542025-04-30T06:14:24.969ZDEBGtest_basic: begin collection from MGS mgs_url = http://[::1]:37822
54552025-04-30T06:14:24.969ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/ignition
54562025-04-30T06:14:24.971ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "e284bba4-5506-44c6-88e8-f7ef3f832495", "content-length": "835", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
54572025-04-30T06:14:24.972ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0
54582025-04-30T06:14:24.973ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "70244d65-925e-49ca-92ce-936d97ff8829", "content-length": "734", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
54592025-04-30T06:14:24.973ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=0
54602025-04-30T06:14:24.975ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "03ed91f6-56e8-497d-924a-339b8ab5e05c", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
54612025-04-30T06:14:24.975ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=1
54622025-04-30T06:14:24.976ZINFOtest_basic (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
54632025-04-30T06:14:24.978ZDEBGtest_basic (oximeter): client request body = None method = GET uri = http://127.0.0.1:34211/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
54642025-04-30T06:14:24.979ZTRACtest_basic (dropshot_internal): incoming request local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 4e144275-560a-4e7b-9820-4d66979732fd uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
54652025-04-30T06:14:24.979ZTRACtest_basic (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 4e144275-560a-4e7b-9820-4d66979732fd resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
54662025-04-30T06:14:24.979ZDEBGtest_basic (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 4e144275-560a-4e7b-9820-4d66979732fd roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
54672025-04-30T06:14:24.982ZDEBGtest_basic (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 4e144275-560a-4e7b-9820-4d66979732fd resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
54682025-04-30T06:14:24.993ZINFOtest_basic (dropshot_internal): request completed latency_us = 14654 local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 4e144275-560a-4e7b-9820-4d66979732fd response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
54692025-04-30T06:14:24.994ZDEBGtest_basic (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:34211/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "4e144275-560a-4e7b-9820-4d66979732fd", "content-length": "405", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
54702025-04-30T06:14:24.994ZDEBGtest_basic (oximeter): client request body = None method = GET uri = http://127.0.0.1:34211/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
54712025-04-30T06:14:24.994ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "68ce1908-4cb1-4408-bfd6-9c39477fdbb8", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
54722025-04-30T06:14:24.995ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=0
54732025-04-30T06:14:24.995ZTRACtest_basic (dropshot_internal): incoming request local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 3849d64a-4263-4c50-b4a5-f444f6823d51 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
54742025-04-30T06:14:24.995ZTRACtest_basic (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 3849d64a-4263-4c50-b4a5-f444f6823d51 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
54752025-04-30T06:14:24.995ZDEBGtest_basic (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 3849d64a-4263-4c50-b4a5-f444f6823d51 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
54762025-04-30T06:14:24.996ZDEBGtest_basic (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 3849d64a-4263-4c50-b4a5-f444f6823d51 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
54772025-04-30T06:14:24.999ZINFOtest_basic (dropshot_internal): request completed latency_us = 4185 local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 3849d64a-4263-4c50-b4a5-f444f6823d51 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
54782025-04-30T06:14:24.999ZDEBGtest_basic (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:34211/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "3849d64a-4263-4c50-b4a5-f444f6823d51", "content-length": "29", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
54792025-04-30T06:14:24.999ZTRACtest_basic (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 088ac96c-981b-4054-bea6-d8ab23ae1366
54802025-04-30T06:14:25.000ZTRACtest_basic (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
54812025-04-30T06:14:25.000ZINFOtest_basic (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
54822025-04-30T06:14:25.001ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1001a519-313d-4743-a037-f5330b7c6384", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
54832025-04-30T06:14:25.001ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=1
54842025-04-30T06:14:25.005ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f0e9fa2d-007d-4f3a-bd1d-bb7af5316fd6", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
54852025-04-30T06:14:25.005ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=0
54862025-04-30T06:14:25.007ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "af2b37d4-c611-4c54-b81c-4d0e621502ed", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
54872025-04-30T06:14:25.008ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=1
54882025-04-30T06:14:25.020ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3bb7ddf0-e294-437a-8264-95228d0121b2", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
54892025-04-30T06:14:25.020ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cmpa
54902025-04-30T06:14:25.022ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "dce54d99-ee9a-463f-ab39-723850cad5f3", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
54912025-04-30T06:14:25.022ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
54922025-04-30T06:14:25.024ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7f394969-e75c-425c-ad96-0d8cbf70b898", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
54932025-04-30T06:14:25.024ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
54942025-04-30T06:14:25.027ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "978ce33d-909d-45d1-9f9c-3df00c677f38", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
54952025-04-30T06:14:25.027ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
54962025-04-30T06:14:25.028ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "348d5a08-73c2-4df3-9a0c-c3e05854c23c", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
54972025-04-30T06:14:25.028ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1
54982025-04-30T06:14:25.029ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7f4e42d1-c562-4dca-8eb4-5ac1a366c795", "content-length": "734", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
54992025-04-30T06:14:25.029ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=0
55002025-04-30T06:14:25.031ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e34652f3-1dde-42da-b475-85a028237ada", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
55012025-04-30T06:14:25.031ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=1
55022025-04-30T06:14:25.033ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bc90dd3a-efe2-470e-91cf-46512c7c6867", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
55032025-04-30T06:14:25.033ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=0
55042025-04-30T06:14:25.035ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1daaa41b-6c87-46f6-806e-ffdb240a145e", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
55052025-04-30T06:14:25.035ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=1
55062025-04-30T06:14:25.037ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b33658fe-5e54-48c4-99f6-7cb15397b7d8", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
55072025-04-30T06:14:25.037ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=0
55082025-04-30T06:14:25.040ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ad027701-8d6f-41c6-9017-94be633df36f", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
55092025-04-30T06:14:25.040ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=1
55102025-04-30T06:14:25.042ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1e16c58f-5586-4ce7-a55a-ed4642cdcdd6", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
55112025-04-30T06:14:25.042ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cmpa
55122025-04-30T06:14:25.044ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1849d64f-be1b-4659-aa3a-b0f9c3672ade", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
55132025-04-30T06:14:25.044ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
55142025-04-30T06:14:25.045ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2bb090f8-b05e-43ea-a228-4170fcd6c1fa", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
55152025-04-30T06:14:25.045ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
55162025-04-30T06:14:25.047ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a34aaf18-3bdb-4883-b7d8-d434f0a7a186", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
55172025-04-30T06:14:25.047ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
55182025-04-30T06:14:25.050ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0711fed6-17ed-4410-bcb5-5f4a9ef54d57", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
55192025-04-30T06:14:25.050ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0
55202025-04-30T06:14:25.052ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0ddca32b-dd5b-41f5-a692-8fbe595c34a0", "content-length": "723", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
55212025-04-30T06:14:25.053ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=0
55222025-04-30T06:14:25.055ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4bdac364-eb30-4716-884a-e4c28582661d", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
55232025-04-30T06:14:25.055ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=1
55242025-04-30T06:14:25.057ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f95b8bc-2e33-4e82-8a4b-5ce6284bfa73", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
55252025-04-30T06:14:25.057ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=0
55262025-04-30T06:14:25.061ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "65800124-c83e-4596-aece-23466ebee3d4", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
55272025-04-30T06:14:25.061ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=1
55282025-04-30T06:14:25.065ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b2ca092b-f93c-4de3-91ab-0a1b3fbbe97d", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
55292025-04-30T06:14:25.066ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=0
55302025-04-30T06:14:25.069ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ce106d96-a329-438f-a17b-3f8ce69d4209", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
55312025-04-30T06:14:25.069ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=1
55322025-04-30T06:14:25.072ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "05090e88-551b-4607-8361-315f3e7f6fec", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
55332025-04-30T06:14:25.072ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cmpa
55342025-04-30T06:14:25.074ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "211fdcf1-eb1e-4226-9b0e-ca538ad39214", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
55352025-04-30T06:14:25.074ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
55362025-04-30T06:14:25.076ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b30599f9-dee2-46a8-aa3f-c72aa238d3f8", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
55372025-04-30T06:14:25.076ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
55382025-04-30T06:14:25.079ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a81aff23-24ed-4358-bc73-3f538d9713c5", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
55392025-04-30T06:14:25.080ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
55402025-04-30T06:14:25.087ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a1719552-7a5e-4a31-8888-cb507b91f9ec", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
55412025-04-30T06:14:25.087ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1
55422025-04-30T06:14:25.107ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6711cb68-c0db-4e62-a568-af802d56ee77", "content-length": "723", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
55432025-04-30T06:14:25.108ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=0
55442025-04-30T06:14:25.117ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "955a7dbe-6393-4953-9428-576d1bfed7f8", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
55452025-04-30T06:14:25.118ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=1
55462025-04-30T06:14:25.124ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "45e60b3e-83e3-4139-8662-16ef8a0e774c", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:24 GMT"} })
55472025-04-30T06:14:25.124ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=0
55482025-04-30T06:14:25.132ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b6067110-6b93-458f-a5b4-e7228282904d", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
55492025-04-30T06:14:25.133ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=1
55502025-04-30T06:14:25.135ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "100fd005-2ff3-4f5e-9963-781d9c14350d", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
55512025-04-30T06:14:25.137ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=0
55522025-04-30T06:14:25.141ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0b4520e5-bfd1-4c73-886f-8b1cd70efe80", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
55532025-04-30T06:14:25.142ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=1
55542025-04-30T06:14:25.146ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a648901f-ca1c-46b7-8638-1230ac12e279", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
55552025-04-30T06:14:25.146ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cmpa
55562025-04-30T06:14:25.148ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9bbacd53-d2b5-47cc-904e-bb17eaab52ee", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
55572025-04-30T06:14:25.148ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
55582025-04-30T06:14:25.151ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f3a90d4d-09a1-46c7-94d4-621e6146cc04", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
55592025-04-30T06:14:25.152ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
55602025-04-30T06:14:25.155ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7e37d8b9-0346-45b7-9d85-7c522b66995c", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
55612025-04-30T06:14:25.155ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
55622025-04-30T06:14:25.157ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "56ba5bf0-1583-4a4d-a907-d47230da327f", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
55632025-04-30T06:14:25.158ZTRACtest_basic: authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet
55642025-04-30T06:14:25.160ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
55652025-04-30T06:14:25.160ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
55662025-04-30T06:14:25.160ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
55672025-04-30T06:14:25.163ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
55682025-04-30T06:14:25.168ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55692025-04-30T06:14:25.176ZDEBGtest_basic: authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet result = Ok(())
55702025-04-30T06:14:25.176ZTRACtest_basic: authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet
55712025-04-30T06:14:25.176ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
55722025-04-30T06:14:25.176ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
55732025-04-30T06:14:25.176ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
55742025-04-30T06:14:25.177ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
55752025-04-30T06:14:25.185ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55762025-04-30T06:14:25.193ZDEBGtest_basic: authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet result = Ok(())
55772025-04-30T06:14:25.193ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
55782025-04-30T06:14:25.193ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
55792025-04-30T06:14:25.194ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
55802025-04-30T06:14:25.244ZDEBGtest_basic: begin collection from Sled Agent sled_agent_url = http://[::1]:42420
55812025-04-30T06:14:25.244ZDEBGtest_basic: client request SledAgent = http://[::1]:42420 body = None method = GET uri = http://[::1]:42420/inventory
55822025-04-30T06:14:25.246ZINFOtest_basic (dropshot): accepted connection local_addr = [::1]:42420 remote_addr = [::1]:40794 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
55832025-04-30T06:14:25.247ZTRACtest_basic (dropshot): incoming request local_addr = [::1]:42420 method = GET remote_addr = [::1]:40794 req_id = 51597906-b5d6-4a6b-be2b-9c7dfae97aab sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
55842025-04-30T06:14:25.247ZINFOtest_basic (dropshot): request completed latency_us = 485 local_addr = [::1]:42420 method = GET remote_addr = [::1]:40794 req_id = 51597906-b5d6-4a6b-be2b-9c7dfae97aab response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
55852025-04-30T06:14:25.248ZDEBGtest_basic: client response SledAgent = http://[::1]:42420 result = Ok(Response { url: "http://[::1]:42420/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "51597906-b5d6-4a6b-be2b-9c7dfae97aab", "content-length": "2769", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
55862025-04-30T06:14:25.251ZDEBGtest_basic: begin collecting all keepers nkeeper_admin_clients = 0
55872025-04-30T06:14:25.252ZDEBGtest_basic: end collecting all keepers nkeeper_admin_clients = 0
55882025-04-30T06:14:25.252ZDEBGtest_basic: finished collection
55892025-04-30T06:14:25.254ZTRACtest_basic: authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory
55902025-04-30T06:14:25.254ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
55912025-04-30T06:14:25.254ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
55922025-04-30T06:14:25.254ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
55932025-04-30T06:14:25.255ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
55942025-04-30T06:14:25.259ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55952025-04-30T06:14:25.263ZDEBGtest_basic: authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory result = Ok(())
55962025-04-30T06:14:25.263ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
55972025-04-30T06:14:25.263ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
55982025-04-30T06:14:25.264ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
55992025-04-30T06:14:25.293ZWARNtest_basic (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
56002025-04-30T06:14:25.293ZWARNtest_basic: inventory collection failed error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
56012025-04-30T06:14:25.298ZTRACtest_basic: authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory
56022025-04-30T06:14:25.298ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
56032025-04-30T06:14:25.298ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
56042025-04-30T06:14:25.298ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
56052025-04-30T06:14:25.299ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
56062025-04-30T06:14:25.302ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56072025-04-30T06:14:25.307ZDEBGtest_basic: authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory result = Ok(())
56082025-04-30T06:14:25.307ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
56092025-04-30T06:14:25.307ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
56102025-04-30T06:14:25.312ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
56112025-04-30T06:14:25.316ZDEBGtest_basic: inventory_prune_one: nothing eligible for removal (too few) candidates = [(efcb0ee5-cafa-4692-9b75-fcee4732bdca, 0)]
56122025-04-30T06:14:25.316ZTRACtest_basic: lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
56132025-04-30T06:14:25.316ZTRACtest_basic (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 18 2a 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .*..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:52156 req_id = 34e9b6f9-01ba-4a39-a33c-04948b262498
56142025-04-30T06:14:25.316ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 6186,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52156 req_id = 34e9b6f9-01ba-4a39-a33c-04948b262498
56152025-04-30T06:14:25.317ZDEBGtest_basic (store): query key key = _mgs._tcp
56162025-04-30T06:14:25.317ZDEBGtest_basic (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
56172025-04-30T06:14:25.317ZDEBGtest_basic (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:52156 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 37822, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 34e9b6f9-01ba-4a39-a33c-04948b262498
56182025-04-30T06:14:25.317ZDEBGtest_basic: lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 37822, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2530, tv_nsec: 116459253 } })
56192025-04-30T06:14:25.317ZTRACtest_basic: lookup_service_targets: looking up SRV target
56202025-04-30T06:14:25.318ZTRACtest_basic (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 41 44 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e AD..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:62957 req_id = ca80e14e-3556-4055-b208-c50323afb0b4
56212025-04-30T06:14:25.318ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 16708,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62957 req_id = ca80e14e-3556-4055-b208-c50323afb0b4
56222025-04-30T06:14:25.318ZDEBGtest_basic (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
56232025-04-30T06:14:25.318ZDEBGtest_basic (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:62957 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = ca80e14e-3556-4055-b208-c50323afb0b4
56242025-04-30T06:14:25.381ZTRACtest_basic: lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
56252025-04-30T06:14:25.381ZDEBGtest_basic (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
56262025-04-30T06:14:25.382ZDEBGtest_basic (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
56272025-04-30T06:14:25.385ZTRACtest_basic (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 8c 20 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c . ..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:61858 req_id = f07e7f55-dda3-4940-b360-1cf981815c86
56282025-04-30T06:14:25.385ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 35872,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:61858 req_id = f07e7f55-dda3-4940-b360-1cf981815c86
56292025-04-30T06:14:25.386ZDEBGtest_basic (store): query key key = _clickhouse-admin-keeper._tcp
56302025-04-30T06:14:25.386ZERROtest_basic (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 35872, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:61858 req_id = f07e7f55-dda3-4940-b360-1cf981815c86
56312025-04-30T06:14:25.386ZTRACtest_basic (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
56322025-04-30T06:14:25.387ZDEBGtest_basic (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
56332025-04-30T06:14:25.387ZTRACtest_basic (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
56342025-04-30T06:14:25.387ZTRACtest_basic (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
56352025-04-30T06:14:25.387ZDEBGtest_basic: begin collection
56362025-04-30T06:14:25.387ZDEBGtest_basic: begin collection from MGS mgs_url = http://[::1]:37822
56372025-04-30T06:14:25.387ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/ignition
56382025-04-30T06:14:25.395ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "4eb8e0f5-a495-4597-ad3e-120442607c6d", "content-length": "835", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
56392025-04-30T06:14:25.396ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0
56402025-04-30T06:14:25.401ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "03e76d10-a068-4f79-bfb5-de4e916081d4", "content-length": "734", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
56412025-04-30T06:14:25.402ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=0
56422025-04-30T06:14:25.406ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0071954-3e3d-49ee-87bd-ea93cd0e2c02", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
56432025-04-30T06:14:25.407ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=1
56442025-04-30T06:14:25.410ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f7d8a229-aa05-4837-8e79-d0a412e9c71f", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
56452025-04-30T06:14:25.410ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=0
56462025-04-30T06:14:25.412ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7dd8cf37-26a2-41de-8c96-ab900b839a2a", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
56472025-04-30T06:14:25.413ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=1
56482025-04-30T06:14:25.417ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5e68d674-2464-4cb2-8792-b4d12428407c", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
56492025-04-30T06:14:25.417ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=0
56502025-04-30T06:14:25.420ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc33ac87-2776-48d5-b7cc-bdb5acb68c01", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
56512025-04-30T06:14:25.420ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=1
56522025-04-30T06:14:25.423ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "77f1db2c-0e3a-42a3-ac8b-7e189afadac3", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
56532025-04-30T06:14:25.424ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cmpa
56542025-04-30T06:14:25.425ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6ecceb6f-cf1f-42d6-ac85-15965dadeec5", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
56552025-04-30T06:14:25.425ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
56562025-04-30T06:14:25.427ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4cbef385-c731-44a2-bd62-8c73fe97fcce", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
56572025-04-30T06:14:25.427ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
56582025-04-30T06:14:25.429ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "867c92a9-f47a-410e-bfa7-f6e8ce82d13a", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
56592025-04-30T06:14:25.429ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
56602025-04-30T06:14:25.430ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3422208a-d2e9-47ac-b4f6-aa98fae3aff8", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
56612025-04-30T06:14:25.430ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1
56622025-04-30T06:14:25.431ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "83e719ed-5b3d-4b77-aba7-d1f81ecf6e5f", "content-length": "734", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
56632025-04-30T06:14:25.432ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=0
56642025-04-30T06:14:25.434ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "db7ac9f1-40ab-406f-9e00-a92beed90ba0", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
56652025-04-30T06:14:25.434ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=1
56662025-04-30T06:14:25.437ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e917d436-c3d6-4d89-9e61-c173fbac73b9", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
56672025-04-30T06:14:25.437ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=0
56682025-04-30T06:14:25.443ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "976b07df-2fb2-4650-a0e9-5b407796512e", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
56692025-04-30T06:14:25.443ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=1
56702025-04-30T06:14:25.447ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "682dc5d9-567a-47ed-b1c3-56ff4b15721b", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
56712025-04-30T06:14:25.448ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=0
56722025-04-30T06:14:25.459ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "53bd4d58-e579-429d-b5a3-84b1ca93c0bd", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
56732025-04-30T06:14:25.462ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=1
56742025-04-30T06:14:25.496ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ba64609b-7b4c-4138-885e-af30d7e4d186", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
56752025-04-30T06:14:25.497ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cmpa
56762025-04-30T06:14:25.498ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e3dc1e9a-fba0-4a06-aefc-b7c84a9f5191", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
56772025-04-30T06:14:25.498ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
56782025-04-30T06:14:25.501ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fae40892-6ff0-4063-ade4-be08c3c14701", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
56792025-04-30T06:14:25.501ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
56802025-04-30T06:14:25.505ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "944d2e01-f9f9-46a2-a851-29dee60f9dd4", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
56812025-04-30T06:14:25.505ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
56822025-04-30T06:14:25.508ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "42cddbda-d084-443e-9373-886daa39e07d", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
56832025-04-30T06:14:25.509ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0
56842025-04-30T06:14:25.511ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3a68970c-4206-4802-b473-8dc27e0749e6", "content-length": "723", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
56852025-04-30T06:14:25.511ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=0
56862025-04-30T06:14:25.516ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "33a43e03-8cce-4cd3-b96a-2f9ad3c45e34", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
56872025-04-30T06:14:25.516ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=1
56882025-04-30T06:14:25.520ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "463caaa1-fd8a-4559-8537-571fb49cd0c2", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
56892025-04-30T06:14:25.520ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=0
56902025-04-30T06:14:25.526ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5ef5974a-6658-4dd1-89b3-5ad3b8df356e", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
56912025-04-30T06:14:25.527ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=1
56922025-04-30T06:14:25.534ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "46a3dfdd-03fd-4760-9690-ce65a23dfa77", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
56932025-04-30T06:14:25.535ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=0
56942025-04-30T06:14:25.535ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "28a188f7-923f-463d-a22c-52af8fb4dbcb", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
56952025-04-30T06:14:25.535ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=1
56962025-04-30T06:14:25.538ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "892aa9b6-0d61-4ae5-9a3a-45d65aa0d359", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
56972025-04-30T06:14:25.538ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cmpa
56982025-04-30T06:14:25.539ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "81c5ba76-b820-44b9-afab-1fc9bc44117a", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
56992025-04-30T06:14:25.540ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
57002025-04-30T06:14:25.553ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a28eef2-2822-4603-8fa1-4b5f6ca50c6c", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
57012025-04-30T06:14:25.553ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
57022025-04-30T06:14:25.554ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "45cbd9d8-644e-4f0b-9888-f169a477c7ed", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
57032025-04-30T06:14:25.554ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
57042025-04-30T06:14:25.556ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "32816286-a871-44b9-b359-c4b6d5c2a751", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
57052025-04-30T06:14:25.556ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1
57062025-04-30T06:14:25.558ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "59abd62b-8740-4a77-8a2c-1ec5c4ca91d6", "content-length": "723", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
57072025-04-30T06:14:25.559ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=0
57082025-04-30T06:14:25.566ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a433782b-0088-4ae6-a1cc-f1e81ccd35e5", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
57092025-04-30T06:14:25.566ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=1
57102025-04-30T06:14:25.570ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3476954a-a064-4b89-924d-5d335619fe6f", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
57112025-04-30T06:14:25.570ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=0
57122025-04-30T06:14:25.572ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e27b9d0b-bec3-48f9-99e0-c438677e64b9", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
57132025-04-30T06:14:25.572ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=1
57142025-04-30T06:14:25.580ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d619b56a-4ad9-491b-a9a0-eae7e89b5ab9", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
57152025-04-30T06:14:25.581ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=0
57162025-04-30T06:14:25.586ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "613c9d1f-113b-4431-8122-82247fa75adf", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
57172025-04-30T06:14:25.586ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=1
57182025-04-30T06:14:25.599ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0b872bb4-69ef-4897-8c35-f153872627e2", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
57192025-04-30T06:14:25.600ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cmpa
57202025-04-30T06:14:25.602ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d49eb1b9-55d3-43e2-81c7-a215c7d12abb", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
57212025-04-30T06:14:25.603ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
57222025-04-30T06:14:25.607ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "96904370-6bea-40f6-b09a-b52ad3ee5d5b", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
57232025-04-30T06:14:25.608ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
57242025-04-30T06:14:25.609ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6d17ae29-fef0-48d9-bb81-eada644a24bb", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
57252025-04-30T06:14:25.611ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
57262025-04-30T06:14:25.611ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cde2841b-8c26-4832-833d-6b5147d1e9dc", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
57272025-04-30T06:14:25.611ZTRACtest_basic: authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet
57282025-04-30T06:14:25.611ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
57292025-04-30T06:14:25.611ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
57302025-04-30T06:14:25.611ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
57312025-04-30T06:14:25.612ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
57322025-04-30T06:14:25.631ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57332025-04-30T06:14:25.649ZDEBGtest_basic: authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet result = Ok(())
57342025-04-30T06:14:25.649ZTRACtest_basic: authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet
57352025-04-30T06:14:25.649ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
57362025-04-30T06:14:25.649ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
57372025-04-30T06:14:25.649ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
57382025-04-30T06:14:25.651ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
57392025-04-30T06:14:25.655ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57402025-04-30T06:14:25.663ZDEBGtest_basic: authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet result = Ok(())
57412025-04-30T06:14:25.664ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
57422025-04-30T06:14:25.664ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
57432025-04-30T06:14:25.664ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
57442025-04-30T06:14:25.733ZDEBGtest_basic: begin collection from Sled Agent sled_agent_url = http://[::1]:42420
57452025-04-30T06:14:25.734ZDEBGtest_basic: client request SledAgent = http://[::1]:42420 body = None method = GET uri = http://[::1]:42420/inventory
57462025-04-30T06:14:25.734ZINFOtest_basic (dropshot): accepted connection local_addr = [::1]:42420 remote_addr = [::1]:56617 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
57472025-04-30T06:14:25.735ZDEBGtest_basic (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
57482025-04-30T06:14:25.735ZDEBGtest_basic (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
57492025-04-30T06:14:25.735ZDEBGtest_basic (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812
57502025-04-30T06:14:25.736ZTRACtest_basic (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
57512025-04-30T06:14:25.736ZTRACtest_basic (dropshot): incoming request local_addr = [::1]:42420 method = GET remote_addr = [::1]:56617 req_id = 2639991c-f81d-46a3-81c0-fcb6112e525f sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
57522025-04-30T06:14:25.737ZINFOtest_basic (dropshot): request completed latency_us = 431 local_addr = [::1]:42420 method = GET remote_addr = [::1]:56617 req_id = 2639991c-f81d-46a3-81c0-fcb6112e525f response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
57532025-04-30T06:14:25.737ZDEBGtest_basic: client response SledAgent = http://[::1]:42420 result = Ok(Response { url: "http://[::1]:42420/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "2639991c-f81d-46a3-81c0-fcb6112e525f", "content-length": "2769", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
57542025-04-30T06:14:25.740ZDEBGtest_basic: begin collecting all keepers nkeeper_admin_clients = 0
57552025-04-30T06:14:25.741ZDEBGtest_basic: end collecting all keepers nkeeper_admin_clients = 0
57562025-04-30T06:14:25.741ZDEBGtest_basic: finished collection
57572025-04-30T06:14:25.743ZTRACtest_basic: authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory
57582025-04-30T06:14:25.743ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
57592025-04-30T06:14:25.743ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
57602025-04-30T06:14:25.743ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
57612025-04-30T06:14:25.744ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
57622025-04-30T06:14:25.748ZDEBGtest_basic (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_rows = 2 table_name = fields_string
57632025-04-30T06:14:25.749ZTRACtest_basic (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
57642025-04-30T06:14:25.749ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57652025-04-30T06:14:25.755ZDEBGtest_basic: authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory result = Ok(())
57662025-04-30T06:14:25.755ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
57672025-04-30T06:14:25.755ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
57682025-04-30T06:14:25.756ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
57692025-04-30T06:14:25.761ZDEBGtest_basic (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_rows = 1 table_name = measurements_i64
57702025-04-30T06:14:25.761ZTRACtest_basic (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
57712025-04-30T06:14:25.794ZWARNtest_basic (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
57722025-04-30T06:14:25.794ZWARNtest_basic: inventory collection failed error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
57732025-04-30T06:14:25.801ZTRACtest_basic: authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory
57742025-04-30T06:14:25.801ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
57752025-04-30T06:14:25.801ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
57762025-04-30T06:14:25.801ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
57772025-04-30T06:14:25.802ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
57782025-04-30T06:14:25.807ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57792025-04-30T06:14:25.817ZDEBGtest_basic: authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory result = Ok(())
57802025-04-30T06:14:25.817ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
57812025-04-30T06:14:25.817ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
57822025-04-30T06:14:25.818ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
57832025-04-30T06:14:25.823ZDEBGtest_basic: inventory_prune_one: nothing eligible for removal (too few) candidates = [(efcb0ee5-cafa-4692-9b75-fcee4732bdca, 0)]
57842025-04-30T06:14:25.823ZTRACtest_basic: lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
57852025-04-30T06:14:25.827ZTRACtest_basic (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 21 31 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 !1..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:54742 req_id = cb4d8877-59ab-4f74-a820-c74319acf862
57862025-04-30T06:14:25.827ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 8497,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54742 req_id = cb4d8877-59ab-4f74-a820-c74319acf862
57872025-04-30T06:14:25.828ZDEBGtest_basic (store): query key key = _mgs._tcp
57882025-04-30T06:14:25.828ZDEBGtest_basic (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
57892025-04-30T06:14:25.828ZDEBGtest_basic (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:54742 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 37822, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = cb4d8877-59ab-4f74-a820-c74319acf862
57902025-04-30T06:14:25.830ZDEBGtest_basic: lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 37822, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2530, tv_nsec: 629265696 } })
57912025-04-30T06:14:25.830ZTRACtest_basic: lookup_service_targets: looking up SRV target
57922025-04-30T06:14:25.834ZTRACtest_basic (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 45 31 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e E1..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:54700 req_id = 9ac838dd-3b6a-4111-b42e-b14a1345dc03
57932025-04-30T06:14:25.834ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 17713,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54700 req_id = 9ac838dd-3b6a-4111-b42e-b14a1345dc03
57942025-04-30T06:14:25.835ZDEBGtest_basic (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
57952025-04-30T06:14:25.835ZDEBGtest_basic (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:54700 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 9ac838dd-3b6a-4111-b42e-b14a1345dc03
57962025-04-30T06:14:25.942ZTRACtest_basic: lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
57972025-04-30T06:14:25.942ZWARNtest_basic (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
57982025-04-30T06:14:25.943ZTRACtest_basic (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 18 20 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c . ..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:61450 req_id = 0a32afa2-e660-4e56-bee8-0f048685363e
57992025-04-30T06:14:25.943ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 6176,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:61450 req_id = 0a32afa2-e660-4e56-bee8-0f048685363e
58002025-04-30T06:14:25.944ZDEBGtest_basic (store): query key key = _clickhouse-admin-keeper._tcp
58012025-04-30T06:14:25.944ZERROtest_basic (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 6176, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:61450 req_id = 0a32afa2-e660-4e56-bee8-0f048685363e
58022025-04-30T06:14:25.944ZDEBGtest_basic: begin collection
58032025-04-30T06:14:25.944ZDEBGtest_basic: begin collection from MGS mgs_url = http://[::1]:37822
58042025-04-30T06:14:25.944ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/ignition
58052025-04-30T06:14:25.948ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "ce328476-3254-4081-b0a5-4afae2e8166d", "content-length": "835", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
58062025-04-30T06:14:25.948ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0
58072025-04-30T06:14:25.950ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "173215a9-4bd4-4e53-9956-06a34aae55ed", "content-length": "734", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
58082025-04-30T06:14:25.950ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=0
58092025-04-30T06:14:25.954ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "93175446-af26-4fda-97f4-96944d33bad1", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
58102025-04-30T06:14:25.954ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=1
58112025-04-30T06:14:25.958ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2de208a5-0482-4ca1-bc9c-6f8c4e0cf88a", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
58122025-04-30T06:14:25.958ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=0
58132025-04-30T06:14:25.960ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7cf2441b-0a30-4b19-9d20-ce0c090cc977", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
58142025-04-30T06:14:25.960ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=1
58152025-04-30T06:14:25.962ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c37bcdf1-92bf-4778-b772-75cc635f1c2a", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
58162025-04-30T06:14:25.962ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=0
58172025-04-30T06:14:25.963ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "74aad65e-1682-4372-888f-5c5bdb699964", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
58182025-04-30T06:14:25.964ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=1
58192025-04-30T06:14:25.966ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "66d703ea-092c-4271-96ed-d3ec411fb096", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
58202025-04-30T06:14:25.966ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cmpa
58212025-04-30T06:14:25.967ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a28c8cb7-5b52-48bf-a699-95a757fcb9e8", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
58222025-04-30T06:14:25.967ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
58232025-04-30T06:14:25.968ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5334e236-b7a6-41fc-b92d-3d33c0a6fdc9", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
58242025-04-30T06:14:25.968ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
58252025-04-30T06:14:25.970ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "83baa31b-60c6-4938-a44b-e83a499fe71a", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
58262025-04-30T06:14:25.970ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
58272025-04-30T06:14:25.971ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0faf04d9-9c98-4d31-bd3c-aedd8873d7f9", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
58282025-04-30T06:14:25.971ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1
58292025-04-30T06:14:25.972ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0b602c85-f9a6-406a-a95e-f200f47656ea", "content-length": "734", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
58302025-04-30T06:14:25.972ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=0
58312025-04-30T06:14:25.975ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "07e95106-20b9-41c3-b40e-5e1e19438f8c", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
58322025-04-30T06:14:25.975ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=1
58332025-04-30T06:14:25.977ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "08e88ae2-89b5-4b8d-b3c9-0a71d7e5c19e", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
58342025-04-30T06:14:25.977ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=0
58352025-04-30T06:14:25.979ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3b102465-9bca-4eda-9ff3-743caa099dbf", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
58362025-04-30T06:14:25.979ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=1
58372025-04-30T06:14:25.981ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "478b68ef-4ede-4680-bef6-031885a65274", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
58382025-04-30T06:14:25.981ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=0
58392025-04-30T06:14:25.983ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "21bed0f4-d1f4-4185-8aa7-2b36da85394c", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
58402025-04-30T06:14:25.983ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=1
58412025-04-30T06:14:25.986ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3fcdf868-bb52-4c59-a263-923b60755670", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
58422025-04-30T06:14:25.986ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cmpa
58432025-04-30T06:14:25.987ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f33b0c8b-8a3d-4511-bc1c-67d11c87ff73", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
58442025-04-30T06:14:25.987ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
58452025-04-30T06:14:25.990ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8c19d414-d4b9-4a0e-948e-8eab0f247384", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
58462025-04-30T06:14:25.990ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
58472025-04-30T06:14:25.992ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "12146057-63da-4b45-9c53-e3686d7c165b", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
58482025-04-30T06:14:25.992ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
58492025-04-30T06:14:25.993ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b81c8661-b529-42b0-9a36-0d7ddb75703d", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
58502025-04-30T06:14:25.994ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0
58512025-04-30T06:14:25.995ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "598a5da1-2c05-469e-861d-5e14531e5a8e", "content-length": "723", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
58522025-04-30T06:14:25.996ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=0
58532025-04-30T06:14:26.003ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4331c93d-fe05-4084-af17-bbdc541a12d6", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
58542025-04-30T06:14:26.003ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=1
58552025-04-30T06:14:26.006ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "028a0ac5-4c78-4307-964b-eb90a45bb73c", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
58562025-04-30T06:14:26.006ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=0
58572025-04-30T06:14:26.008ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1955e39c-d557-45cd-aee7-d970f0b9af55", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
58582025-04-30T06:14:26.008ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=1
58592025-04-30T06:14:26.010ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a8185a21-3a37-477a-9030-94af249a3552", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
58602025-04-30T06:14:26.010ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=0
58612025-04-30T06:14:26.011ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cf4c71a4-48f2-4689-aa84-bb1bc80b0612", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
58622025-04-30T06:14:26.012ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=1
58632025-04-30T06:14:26.014ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c636ee26-38ec-404e-86d8-a9e61d47af57", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
58642025-04-30T06:14:26.014ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cmpa
58652025-04-30T06:14:26.016ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "75ceb6ed-766c-4a5e-bf99-e79a02b8b759", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
58662025-04-30T06:14:26.016ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
58672025-04-30T06:14:26.018ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7d050913-2497-47fd-9700-84e582a13d99", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
58682025-04-30T06:14:26.020ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
58692025-04-30T06:14:26.022ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "95cc878a-f8ad-4d55-8260-2a09f1452f45", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
58702025-04-30T06:14:26.023ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
58712025-04-30T06:14:26.024ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "501e0d50-0936-4fb4-8b91-61452b29a44b", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
58722025-04-30T06:14:26.024ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1
58732025-04-30T06:14:26.025ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1932fc9a-a22a-47de-a2e1-9c18a6d6aff4", "content-length": "723", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
58742025-04-30T06:14:26.025ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=0
58752025-04-30T06:14:26.029ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "241d12a6-80bf-4064-a4f5-c6f6d7410ed8", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
58762025-04-30T06:14:26.029ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=1
58772025-04-30T06:14:26.031ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c6f0469b-6b8a-4fca-a02b-6ca311ca9911", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
58782025-04-30T06:14:26.031ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=0
58792025-04-30T06:14:26.034ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "506cb543-758e-4171-97a6-8d952007d8a0", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
58802025-04-30T06:14:26.034ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=1
58812025-04-30T06:14:26.037ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6bd9b926-1b24-4c10-8cb9-18e57e6c27d2", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
58822025-04-30T06:14:26.037ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=0
58832025-04-30T06:14:26.038ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6981c40c-735b-467b-92ee-d662c738fc79", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
58842025-04-30T06:14:26.040ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=1
58852025-04-30T06:14:26.045ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fdc59898-d3a1-4f83-90b8-d37171c1534b", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
58862025-04-30T06:14:26.045ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cmpa
58872025-04-30T06:14:26.046ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "87062fca-968f-4468-968b-bd713db67adb", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
58882025-04-30T06:14:26.046ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
58892025-04-30T06:14:26.048ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2914a036-a46d-44d6-87c5-5d4e36fa0416", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
58902025-04-30T06:14:26.048ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
58912025-04-30T06:14:26.050ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c89da9df-d118-42aa-9691-bf7cf77134be", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
58922025-04-30T06:14:26.051ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
58932025-04-30T06:14:26.052ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0bf20a14-7224-4be9-ada1-9455612878d9", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:25 GMT"} })
58942025-04-30T06:14:26.052ZTRACtest_basic: authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet
58952025-04-30T06:14:26.052ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
58962025-04-30T06:14:26.052ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
58972025-04-30T06:14:26.052ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
58982025-04-30T06:14:26.053ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
58992025-04-30T06:14:26.059ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59002025-04-30T06:14:26.069ZDEBGtest_basic: authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet result = Ok(())
59012025-04-30T06:14:26.070ZTRACtest_basic: authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet
59022025-04-30T06:14:26.070ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
59032025-04-30T06:14:26.070ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
59042025-04-30T06:14:26.070ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
59052025-04-30T06:14:26.070ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
59062025-04-30T06:14:26.077ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59072025-04-30T06:14:26.091ZDEBGtest_basic: authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet result = Ok(())
59082025-04-30T06:14:26.091ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
59092025-04-30T06:14:26.091ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
59102025-04-30T06:14:26.093ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
59112025-04-30T06:14:26.179ZDEBGtest_basic: begin collection from Sled Agent sled_agent_url = http://[::1]:42420
59122025-04-30T06:14:26.179ZDEBGtest_basic: client request SledAgent = http://[::1]:42420 body = None method = GET uri = http://[::1]:42420/inventory
59132025-04-30T06:14:26.182ZINFOtest_basic (dropshot): accepted connection local_addr = [::1]:42420 remote_addr = [::1]:42667 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
59142025-04-30T06:14:26.186ZTRACtest_basic (dropshot): incoming request local_addr = [::1]:42420 method = GET remote_addr = [::1]:42667 req_id = 4051df42-3638-45b9-83c1-5a1e6d3b37c1 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
59152025-04-30T06:14:26.189ZINFOtest_basic (dropshot): request completed latency_us = 2767 local_addr = [::1]:42420 method = GET remote_addr = [::1]:42667 req_id = 4051df42-3638-45b9-83c1-5a1e6d3b37c1 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
59162025-04-30T06:14:26.190ZDEBGtest_basic: client response SledAgent = http://[::1]:42420 result = Ok(Response { url: "http://[::1]:42420/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "4051df42-3638-45b9-83c1-5a1e6d3b37c1", "content-length": "2769", "date": "Wed, 30 Apr 2025 06:14:26 GMT"} })
59172025-04-30T06:14:26.195ZDEBGtest_basic: begin collecting all keepers nkeeper_admin_clients = 0
59182025-04-30T06:14:26.195ZDEBGtest_basic: end collecting all keepers nkeeper_admin_clients = 0
59192025-04-30T06:14:26.195ZDEBGtest_basic: finished collection
59202025-04-30T06:14:26.197ZTRACtest_basic: authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory
59212025-04-30T06:14:26.197ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
59222025-04-30T06:14:26.197ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
59232025-04-30T06:14:26.198ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
59242025-04-30T06:14:26.198ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
59252025-04-30T06:14:26.213ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59262025-04-30T06:14:26.220ZDEBGtest_basic: authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory result = Ok(())
59272025-04-30T06:14:26.221ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
59282025-04-30T06:14:26.221ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
59292025-04-30T06:14:26.222ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
59302025-04-30T06:14:26.371ZDEBGtest_basic (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
59312025-04-30T06:14:26.371ZDEBGtest_basic (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
59322025-04-30T06:14:26.383ZTRACtest_basic (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
59332025-04-30T06:14:26.385ZDEBGtest_basic (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
59342025-04-30T06:14:26.385ZTRACtest_basic (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
59352025-04-30T06:14:26.385ZTRACtest_basic (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
59362025-04-30T06:14:26.404ZWARNtest_basic (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
59372025-04-30T06:14:26.404ZWARNtest_basic: inventory collection failed error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
59382025-04-30T06:14:26.416ZTRACtest_basic: authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory
59392025-04-30T06:14:26.417ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
59402025-04-30T06:14:26.417ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
59412025-04-30T06:14:26.418ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
59422025-04-30T06:14:26.422ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
59432025-04-30T06:14:26.443ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59442025-04-30T06:14:26.456ZDEBGtest_basic: authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory result = Ok(())
59452025-04-30T06:14:26.456ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
59462025-04-30T06:14:26.456ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
59472025-04-30T06:14:26.460ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
59482025-04-30T06:14:26.464ZDEBGtest_basic: inventory_prune_one: nothing eligible for removal (too few) candidates = [(efcb0ee5-cafa-4692-9b75-fcee4732bdca, 0)]
59492025-04-30T06:14:26.464ZTRACtest_basic: lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
59502025-04-30T06:14:26.465ZTRACtest_basic (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 16 e1 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:62770 req_id = cbb776f7-45b8-429a-bd5b-39c421d88c60
59512025-04-30T06:14:26.465ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 5857,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62770 req_id = cbb776f7-45b8-429a-bd5b-39c421d88c60
59522025-04-30T06:14:26.465ZDEBGtest_basic (store): query key key = _mgs._tcp
59532025-04-30T06:14:26.465ZDEBGtest_basic (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
59542025-04-30T06:14:26.465ZDEBGtest_basic (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:62770 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 37822, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = cbb776f7-45b8-429a-bd5b-39c421d88c60
59552025-04-30T06:14:26.466ZDEBGtest_basic: lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 37822, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2531, tv_nsec: 265068422 } })
59562025-04-30T06:14:26.466ZTRACtest_basic: lookup_service_targets: looking up SRV target
59572025-04-30T06:14:26.466ZTRACtest_basic (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 76 0a 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e v...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:63449 req_id = 606359b8-b588-49df-af95-39e8b505f189
59582025-04-30T06:14:26.466ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 30218,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63449 req_id = 606359b8-b588-49df-af95-39e8b505f189
59592025-04-30T06:14:26.466ZDEBGtest_basic (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
59602025-04-30T06:14:26.466ZDEBGtest_basic (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:63449 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 606359b8-b588-49df-af95-39e8b505f189
59612025-04-30T06:14:26.634ZTRACtest_basic: lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
59622025-04-30T06:14:26.634ZTRACtest_basic (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 7f 46 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c .F..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:60844 req_id = 9072c00b-8782-4bcb-b27e-0662c10a45e6
59632025-04-30T06:14:26.635ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 32582,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60844 req_id = 9072c00b-8782-4bcb-b27e-0662c10a45e6
59642025-04-30T06:14:26.635ZDEBGtest_basic (store): query key key = _clickhouse-admin-keeper._tcp
59652025-04-30T06:14:26.635ZERROtest_basic (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 32582, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:60844 req_id = 9072c00b-8782-4bcb-b27e-0662c10a45e6
59662025-04-30T06:14:26.635ZDEBGtest_basic: begin collection
59672025-04-30T06:14:26.636ZDEBGtest_basic: begin collection from MGS mgs_url = http://[::1]:37822
59682025-04-30T06:14:26.636ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/ignition
59692025-04-30T06:14:26.643ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "e6b8af11-5a79-4573-baf8-cb9a4e70444e", "content-length": "835", "date": "Wed, 30 Apr 2025 06:14:26 GMT"} })
59702025-04-30T06:14:26.643ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0
59712025-04-30T06:14:26.645ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "41a5ad86-74fb-4d28-9417-a298ee0ba57d", "content-length": "734", "date": "Wed, 30 Apr 2025 06:14:26 GMT"} })
59722025-04-30T06:14:26.646ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=0
59732025-04-30T06:14:26.652ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "817c7a86-ac69-4391-94f2-36c3b629c6aa", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:26 GMT"} })
59742025-04-30T06:14:26.653ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=1
59752025-04-30T06:14:26.660ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "55d77061-bede-43c7-bc0c-c7325ba267ba", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:26 GMT"} })
59762025-04-30T06:14:26.660ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=0
59772025-04-30T06:14:26.663ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "42a0b460-2a4c-4e90-a5cd-1781eb3e6fd5", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:26 GMT"} })
59782025-04-30T06:14:26.672ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=1
59792025-04-30T06:14:26.679ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d5b22c06-f5bb-4f8f-966f-6eb9a1469fca", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:26 GMT"} })
59802025-04-30T06:14:26.679ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=0
59812025-04-30T06:14:26.686ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e1a54b8b-d8c5-4a49-81d5-097941ba756c", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:26 GMT"} })
59822025-04-30T06:14:26.687ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=1
59832025-04-30T06:14:26.695ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "379f70d5-57b4-4dce-aaa6-dd8e6699108f", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:26 GMT"} })
59842025-04-30T06:14:26.695ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cmpa
59852025-04-30T06:14:26.696ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "799b9d68-7bff-456b-8d1d-a62ade62520e", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:26 GMT"} })
59862025-04-30T06:14:26.696ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
59872025-04-30T06:14:26.697ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1b20147f-3149-4856-8c20-4e944e0abf3e", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:26 GMT"} })
59882025-04-30T06:14:26.697ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
59892025-04-30T06:14:26.698ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8f012a6b-45bf-4b43-8841-0876d35f4588", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:26 GMT"} })
59902025-04-30T06:14:26.699ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
59912025-04-30T06:14:26.700ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9ca31262-45b4-4395-a43c-6067eec15bff", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:26 GMT"} })
59922025-04-30T06:14:26.700ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1
59932025-04-30T06:14:26.701ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5d36cc5a-9b8f-47bc-897e-c04512a791fc", "content-length": "734", "date": "Wed, 30 Apr 2025 06:14:26 GMT"} })
59942025-04-30T06:14:26.701ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=0
59952025-04-30T06:14:26.703ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2a01ef0e-d986-43c0-a089-bbe64abb97e1", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:26 GMT"} })
59962025-04-30T06:14:26.704ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=1
59972025-04-30T06:14:26.708ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e69d47db-f75d-4daf-8227-441270b55b86", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:26 GMT"} })
59982025-04-30T06:14:26.708ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=0
59992025-04-30T06:14:26.710ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "740c809f-d81e-4d8b-9831-5c0984c92da7", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:26 GMT"} })
60002025-04-30T06:14:26.710ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=1
60012025-04-30T06:14:26.712ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c133f59d-3839-4ea2-a4c8-063e1770beda", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:26 GMT"} })
60022025-04-30T06:14:26.712ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=0
60032025-04-30T06:14:26.717ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6fb1b901-af0a-4333-a3c4-c45937854dd0", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:26 GMT"} })
60042025-04-30T06:14:26.717ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=1
60052025-04-30T06:14:26.724ZDEBGtest_basic (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
60062025-04-30T06:14:26.724ZDEBGtest_basic (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
60072025-04-30T06:14:26.724ZDEBGtest_basic (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812
60082025-04-30T06:14:26.729ZTRACtest_basic (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
60092025-04-30T06:14:26.734ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "daf0023f-da49-4c8f-92b4-3b8973386d13", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:26 GMT"} })
60102025-04-30T06:14:26.735ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cmpa
60112025-04-30T06:14:26.739ZDEBGtest_basic (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_rows = 2 table_name = fields_string
60122025-04-30T06:14:26.739ZTRACtest_basic (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
60132025-04-30T06:14:26.741ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cfcf4959-e6cf-4e05-9fbc-d14aa39dc29e", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:26 GMT"} })
60142025-04-30T06:14:26.742ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
60152025-04-30T06:14:26.744ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1423b5c0-9820-4714-8a03-aaaa52acafb8", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:26 GMT"} })
60162025-04-30T06:14:26.745ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
60172025-04-30T06:14:26.745ZDEBGtest_basic (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_rows = 1 table_name = measurements_i64
60182025-04-30T06:14:26.745ZTRACtest_basic (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
60192025-04-30T06:14:26.747ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "bc23e7aa-3c3f-4667-a806-9230ad4a029c", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:26 GMT"} })
60202025-04-30T06:14:26.747ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
60212025-04-30T06:14:26.748ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ab50f0f1-8276-4065-99cc-cffaa3e52cfd", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:26 GMT"} })
60222025-04-30T06:14:26.748ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0
60232025-04-30T06:14:26.754ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "afdcc86c-0bb1-4dd0-ad04-72cb044bd7ee", "content-length": "723", "date": "Wed, 30 Apr 2025 06:14:26 GMT"} })
60242025-04-30T06:14:26.755ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=0
60252025-04-30T06:14:26.771ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6573c8f9-ac46-4add-8501-8bd55c451986", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:26 GMT"} })
60262025-04-30T06:14:26.772ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=1
60272025-04-30T06:14:26.778ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7b893bb1-2618-4663-921f-5473065a2116", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:26 GMT"} })
60282025-04-30T06:14:26.778ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=0
60292025-04-30T06:14:26.782ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "efa465a4-6f68-455b-803b-312ffb1b72f9", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:26 GMT"} })
60302025-04-30T06:14:26.782ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=1
60312025-04-30T06:14:26.785ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "61d74e3e-7372-42bc-b31a-bdfef505b982", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:26 GMT"} })
60322025-04-30T06:14:26.785ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=0
60332025-04-30T06:14:26.791ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "38a68934-0199-4035-b6af-d1b64614aeee", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:26 GMT"} })
60342025-04-30T06:14:26.791ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=1
60352025-04-30T06:14:26.794ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "da9e2c29-9307-442b-ad2d-63621cd3638c", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:26 GMT"} })
60362025-04-30T06:14:26.795ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cmpa
60372025-04-30T06:14:26.796ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3843afa9-abc8-493f-b3c4-3b690d1dfba0", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:26 GMT"} })
60382025-04-30T06:14:26.797ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
60392025-04-30T06:14:26.799ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "26b327fd-a525-4987-8f8a-1f9bf92750c7", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:26 GMT"} })
60402025-04-30T06:14:26.800ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
60412025-04-30T06:14:26.801ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6b67cde1-5f87-4c1c-b67f-7372a35dbc8d", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:26 GMT"} })
60422025-04-30T06:14:26.805ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
60432025-04-30T06:14:26.807ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "70c175f4-177b-4975-a47f-85a23ded80ca", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:26 GMT"} })
60442025-04-30T06:14:26.809ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1
60452025-04-30T06:14:26.811ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fcdd9bd5-0eed-489d-8175-16b8b9edc096", "content-length": "723", "date": "Wed, 30 Apr 2025 06:14:26 GMT"} })
60462025-04-30T06:14:26.812ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=0
60472025-04-30T06:14:26.814ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "60225a48-d396-4ad7-87a4-82e0dad08b35", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:26 GMT"} })
60482025-04-30T06:14:26.814ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=1
60492025-04-30T06:14:26.817ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a1b0e785-bfd9-41e6-b2a3-56d2265fd666", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:26 GMT"} })
60502025-04-30T06:14:26.817ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=0
60512025-04-30T06:14:26.819ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "254f12dc-8f5e-4551-a4ec-5b4e4540319e", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:26 GMT"} })
60522025-04-30T06:14:26.819ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=1
60532025-04-30T06:14:26.821ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ff7dcb83-2a96-4859-8607-ed110c5931aa", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:26 GMT"} })
60542025-04-30T06:14:26.821ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=0
60552025-04-30T06:14:26.823ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ea3d333a-7898-4ee1-8f38-a3b6f68dfff4", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:26 GMT"} })
60562025-04-30T06:14:26.823ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=1
60572025-04-30T06:14:26.825ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "39f4d834-882c-40dc-bce4-03b1ea421ff6", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:26 GMT"} })
60582025-04-30T06:14:26.825ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cmpa
60592025-04-30T06:14:26.825ZWARNtest_basic (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
60602025-04-30T06:14:26.828ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0a5f30fb-5d51-4314-997b-4c9387e5f644", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:26 GMT"} })
60612025-04-30T06:14:26.828ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
60622025-04-30T06:14:26.830ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1fb22497-a51c-46ad-8dac-8abbaae133e5", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:26 GMT"} })
60632025-04-30T06:14:26.830ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
60642025-04-30T06:14:26.834ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0cc7e384-4c76-482e-9076-fcb59993acfe", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:26 GMT"} })
60652025-04-30T06:14:26.834ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
60662025-04-30T06:14:26.835ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0478171f-3c90-4a28-b691-86a651ff909f", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:26 GMT"} })
60672025-04-30T06:14:26.835ZTRACtest_basic: authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet
60682025-04-30T06:14:26.835ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
60692025-04-30T06:14:26.835ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
60702025-04-30T06:14:26.835ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
60712025-04-30T06:14:26.836ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
60722025-04-30T06:14:26.847ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60732025-04-30T06:14:26.859ZDEBGtest_basic: authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet result = Ok(())
60742025-04-30T06:14:26.859ZTRACtest_basic: authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet
60752025-04-30T06:14:26.859ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
60762025-04-30T06:14:26.859ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
60772025-04-30T06:14:26.859ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
60782025-04-30T06:14:26.871ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
60792025-04-30T06:14:26.875ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60802025-04-30T06:14:26.880ZDEBGtest_basic: authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet result = Ok(())
60812025-04-30T06:14:26.880ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
60822025-04-30T06:14:26.880ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
60832025-04-30T06:14:26.881ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
60842025-04-30T06:14:26.982ZDEBGtest_basic: begin collection from Sled Agent sled_agent_url = http://[::1]:42420
60852025-04-30T06:14:26.993ZDEBGtest_basic: client request SledAgent = http://[::1]:42420 body = None method = GET uri = http://[::1]:42420/inventory
60862025-04-30T06:14:26.995ZINFOtest_basic (dropshot): accepted connection local_addr = [::1]:42420 remote_addr = [::1]:52233 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
60872025-04-30T06:14:26.995ZINFOtest_basic (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
60882025-04-30T06:14:26.996ZDEBGtest_basic (oximeter): client request body = None method = GET uri = http://127.0.0.1:34211/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
60892025-04-30T06:14:26.998ZTRACtest_basic (dropshot_internal): incoming request local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 7b490d9b-3a00-47f0-b500-53461a51a29d uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
60902025-04-30T06:14:26.999ZTRACtest_basic (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 7b490d9b-3a00-47f0-b500-53461a51a29d resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
60912025-04-30T06:14:27.005ZDEBGtest_basic (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 7b490d9b-3a00-47f0-b500-53461a51a29d roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
60922025-04-30T06:14:27.017ZDEBGtest_basic (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 7b490d9b-3a00-47f0-b500-53461a51a29d resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
60932025-04-30T06:14:27.019ZTRACtest_basic (dropshot): incoming request local_addr = [::1]:42420 method = GET remote_addr = [::1]:52233 req_id = e8cc8c81-3513-4770-976a-4b927d98d18b sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
60942025-04-30T06:14:27.021ZINFOtest_basic (dropshot): request completed latency_us = 1885 local_addr = [::1]:42420 method = GET remote_addr = [::1]:52233 req_id = e8cc8c81-3513-4770-976a-4b927d98d18b response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
60952025-04-30T06:14:27.022ZDEBGtest_basic: client response SledAgent = http://[::1]:42420 result = Ok(Response { url: "http://[::1]:42420/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "e8cc8c81-3513-4770-976a-4b927d98d18b", "content-length": "2769", "date": "Wed, 30 Apr 2025 06:14:26 GMT"} })
60962025-04-30T06:14:27.026ZDEBGtest_basic: begin collecting all keepers nkeeper_admin_clients = 0
60972025-04-30T06:14:27.026ZDEBGtest_basic: end collecting all keepers nkeeper_admin_clients = 0
60982025-04-30T06:14:27.026ZDEBGtest_basic: finished collection
60992025-04-30T06:14:27.028ZTRACtest_basic: authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory
61002025-04-30T06:14:27.028ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
61012025-04-30T06:14:27.028ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
61022025-04-30T06:14:27.028ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
61032025-04-30T06:14:27.028ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
61042025-04-30T06:14:27.030ZINFOtest_basic (dropshot_internal): request completed latency_us = 32518 local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 7b490d9b-3a00-47f0-b500-53461a51a29d response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
61052025-04-30T06:14:27.032ZDEBGtest_basic (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:34211/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "7b490d9b-3a00-47f0-b500-53461a51a29d", "content-length": "405", "date": "Wed, 30 Apr 2025 06:14:26 GMT"} })
61062025-04-30T06:14:27.032ZDEBGtest_basic (oximeter): client request body = None method = GET uri = http://127.0.0.1:34211/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
61072025-04-30T06:14:27.032ZTRACtest_basic (dropshot_internal): incoming request local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = c3568c83-a460-4578-8001-fd30afa0127f uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
61082025-04-30T06:14:27.033ZTRACtest_basic (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = c3568c83-a460-4578-8001-fd30afa0127f resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
61092025-04-30T06:14:27.033ZDEBGtest_basic (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = c3568c83-a460-4578-8001-fd30afa0127f roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
61102025-04-30T06:14:27.033ZDEBGtest_basic (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = c3568c83-a460-4578-8001-fd30afa0127f resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
61112025-04-30T06:14:27.033ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61122025-04-30T06:14:27.046ZDEBGtest_basic: authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory result = Ok(())
61132025-04-30T06:14:27.046ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
61142025-04-30T06:14:27.046ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
61152025-04-30T06:14:27.047ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
61162025-04-30T06:14:27.052ZINFOtest_basic (dropshot_internal): request completed latency_us = 19688 local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = c3568c83-a460-4578-8001-fd30afa0127f response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
61172025-04-30T06:14:27.053ZDEBGtest_basic (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:34211/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "c3568c83-a460-4578-8001-fd30afa0127f", "content-length": "29", "date": "Wed, 30 Apr 2025 06:14:26 GMT"} })
61182025-04-30T06:14:27.055ZTRACtest_basic (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 088ac96c-981b-4054-bea6-d8ab23ae1366
61192025-04-30T06:14:27.055ZTRACtest_basic (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
61202025-04-30T06:14:27.055ZINFOtest_basic (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
61212025-04-30T06:14:27.149ZWARNtest_basic (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
61222025-04-30T06:14:27.149ZWARNtest_basic: inventory collection failed error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
61232025-04-30T06:14:27.156ZTRACtest_basic: authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory
61242025-04-30T06:14:27.156ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
61252025-04-30T06:14:27.156ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
61262025-04-30T06:14:27.156ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
61272025-04-30T06:14:27.157ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
61282025-04-30T06:14:27.170ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61292025-04-30T06:14:27.177ZDEBGtest_basic: authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory result = Ok(())
61302025-04-30T06:14:27.177ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
61312025-04-30T06:14:27.177ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
61322025-04-30T06:14:27.178ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
61332025-04-30T06:14:27.182ZDEBGtest_basic: inventory_prune_one: nothing eligible for removal (too few) candidates = [(efcb0ee5-cafa-4692-9b75-fcee4732bdca, 0)]
61342025-04-30T06:14:27.182ZTRACtest_basic: lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
61352025-04-30T06:14:27.200ZTRACtest_basic (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: ed a1 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:64056 req_id = 59e6984e-6f7b-4a5d-9680-9a07dbad8d8d
61362025-04-30T06:14:27.201ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 60833,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64056 req_id = 59e6984e-6f7b-4a5d-9680-9a07dbad8d8d
61372025-04-30T06:14:27.201ZDEBGtest_basic (store): query key key = _mgs._tcp
61382025-04-30T06:14:27.201ZDEBGtest_basic (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
61392025-04-30T06:14:27.201ZDEBGtest_basic (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:64056 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 37822, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 59e6984e-6f7b-4a5d-9680-9a07dbad8d8d
61402025-04-30T06:14:27.202ZDEBGtest_basic: lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 37822, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2532, tv_nsec: 1710603 } })
61412025-04-30T06:14:27.202ZTRACtest_basic: lookup_service_targets: looking up SRV target
61422025-04-30T06:14:27.203ZTRACtest_basic (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 27 95 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e '...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:49856 req_id = 4f2eaf34-d51e-4c10-9d48-745edf0364c8
61432025-04-30T06:14:27.203ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 10133,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:49856 req_id = 4f2eaf34-d51e-4c10-9d48-745edf0364c8
61442025-04-30T06:14:27.203ZDEBGtest_basic (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
61452025-04-30T06:14:27.203ZDEBGtest_basic (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:49856 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 4f2eaf34-d51e-4c10-9d48-745edf0364c8
61462025-04-30T06:14:27.322ZTRACtest_basic: lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
61472025-04-30T06:14:27.322ZDEBGtest_basic (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 088ac96c-981b-4054-bea6-d8ab23ae1366
61482025-04-30T06:14:27.323ZDEBGtest_basic (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 088ac96c-981b-4054-bea6-d8ab23ae1366
61492025-04-30T06:14:27.324ZINFOtest_basic (dropshot): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:1025 local_addr = 127.0.0.1:46907 remote_addr = 127.0.0.1:59737
61502025-04-30T06:14:27.324ZTRACtest_basic (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: cc 95 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:65484 req_id = 108d4a91-3c8c-4d88-bb26-2b4c5226cd08
61512025-04-30T06:14:27.324ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 52373,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:65484 req_id = 108d4a91-3c8c-4d88-bb26-2b4c5226cd08
61522025-04-30T06:14:27.324ZDEBGtest_basic (store): query key key = _clickhouse-admin-keeper._tcp
61532025-04-30T06:14:27.324ZERROtest_basic (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 52373, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:65484 req_id = 108d4a91-3c8c-4d88-bb26-2b4c5226cd08
61542025-04-30T06:14:27.325ZDEBGtest_basic: begin collection
61552025-04-30T06:14:27.325ZDEBGtest_basic: begin collection from MGS mgs_url = http://[::1]:37822
61562025-04-30T06:14:27.325ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/ignition
61572025-04-30T06:14:27.326ZTRACtest_basic (dropshot): incoming request local_addr = 127.0.0.1:46907 method = GET remote_addr = 127.0.0.1:59737 req_id = 2df10c84-6c26-4521-bf5d-69c9be29774e uri = /088ac96c-981b-4054-bea6-d8ab23ae1366
61582025-04-30T06:14:27.367ZINFOtest_basic (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 41893 local_addr = 127.0.0.1:46907 method = GET remote_addr = 127.0.0.1:59737 req_id = 2df10c84-6c26-4521-bf5d-69c9be29774e response_code = 200 uri = /088ac96c-981b-4054-bea6-d8ab23ae1366
61592025-04-30T06:14:27.368ZDEBGtest_basic (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
61602025-04-30T06:14:27.368ZDEBGtest_basic (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
61612025-04-30T06:14:27.369ZTRACtest_basic (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 088ac96c-981b-4054-bea6-d8ab23ae1366
61622025-04-30T06:14:27.390ZDEBGtest_basic (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 5 producer_id = 088ac96c-981b-4054-bea6-d8ab23ae1366
61632025-04-30T06:14:27.390ZTRACtest_basic (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 088ac96c-981b-4054-bea6-d8ab23ae1366
61642025-04-30T06:14:27.390ZTRACtest_basic (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 088ac96c-981b-4054-bea6-d8ab23ae1366
61652025-04-30T06:14:27.391ZDEBGtest_basic (results-sink): inserting 17 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
61662025-04-30T06:14:27.391ZDEBGtest_basic (results-sink-cluster): inserting 17 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
61672025-04-30T06:14:27.391ZDEBGtest_basic (clickhouse-client): unrolling 17 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812
61682025-04-30T06:14:27.392ZDEBGtest_basic (clickhouse-client): retrieving timeseries schema from database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812
61692025-04-30T06:14:27.392ZTRACtest_basic (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 sql = SELECT * FROM oximeter.timeseries_schema WHERE timeseries_name NOT IN ('integration_target:integration_metric') FORMAT Native;
61702025-04-30T06:14:27.397ZTRACtest_basic (clickhouse-client): no new timeseries schema in database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812
61712025-04-30T06:14:27.397ZDEBGtest_basic (clickhouse-client): new timeseries schema collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 schema = TimeseriesSchema { timeseries_name: TimeseriesName("http_service:request_latency_histogram"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "name", field_type: String, source: Target, description: "" }, FieldSchema { name: "operation_id", field_type: String, source: Metric, description: "" }, FieldSchema { name: "status_code", field_type: U16, source: Metric, description: "" }}, datum_type: HistogramU64, version: 1, authz_scope: Fleet, units: Count, created: 2025-04-30T06:14:27.392579077Z } timeseries_name = http_service:request_latency_histogram
61722025-04-30T06:14:27.435ZDEBGtest_basic (clickhouse-client): retrieving timeseries schema from database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812
61732025-04-30T06:14:27.436ZTRACtest_basic (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 sql = SELECT * FROM oximeter.timeseries_schema WHERE timeseries_name NOT IN ('http_service:request_latency_histogram', 'integration_target:integration_metric') FORMAT Native;
61742025-04-30T06:14:27.436ZTRACtest_basic (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
61752025-04-30T06:14:27.437ZDEBGtest_basic (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
61762025-04-30T06:14:27.437ZTRACtest_basic (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
61772025-04-30T06:14:27.437ZTRACtest_basic (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
61782025-04-30T06:14:27.439ZTRACtest_basic (clickhouse-client): no new timeseries schema in database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812
61792025-04-30T06:14:27.439ZDEBGtest_basic (clickhouse-client): new timeseries schema collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 schema = TimeseriesSchema { timeseries_name: TimeseriesName("collection_target:cpus_provisioned"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "id", field_type: Uuid, source: Target, description: "" }}, datum_type: I64, version: 1, authz_scope: Fleet, units: Count, created: 2025-04-30T06:14:27.435784762Z } timeseries_name = collection_target:cpus_provisioned
61802025-04-30T06:14:27.440ZDEBGtest_basic (clickhouse-client): retrieving timeseries schema from database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812
61812025-04-30T06:14:27.440ZTRACtest_basic (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 sql = SELECT * FROM oximeter.timeseries_schema WHERE timeseries_name NOT IN ('collection_target:cpus_provisioned', 'http_service:request_latency_histogram', 'integration_target:integration_metric') FORMAT Native;
61822025-04-30T06:14:27.440ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "2e9b18e1-cd9d-4629-926f-3525eb99c78e", "content-length": "835", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
61832025-04-30T06:14:27.441ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0
61842025-04-30T06:14:27.443ZTRACtest_basic (clickhouse-client): no new timeseries schema in database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812
61852025-04-30T06:14:27.443ZDEBGtest_basic (clickhouse-client): new timeseries schema collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 schema = TimeseriesSchema { timeseries_name: TimeseriesName("collection_target:ram_provisioned"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "id", field_type: Uuid, source: Target, description: "" }}, datum_type: I64, version: 1, authz_scope: Fleet, units: Count, created: 2025-04-30T06:14:27.440319407Z } timeseries_name = collection_target:ram_provisioned
61862025-04-30T06:14:27.443ZDEBGtest_basic (clickhouse-client): retrieving timeseries schema from database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812
61872025-04-30T06:14:27.443ZTRACtest_basic (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 sql = SELECT * FROM oximeter.timeseries_schema WHERE timeseries_name NOT IN ('collection_target:cpus_provisioned', 'collection_target:ram_provisioned', 'http_service:request_latency_histogram', 'integration_target:integration_metric') FORMAT Native;
61882025-04-30T06:14:27.447ZTRACtest_basic (clickhouse-client): no new timeseries schema in database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812
61892025-04-30T06:14:27.447ZDEBGtest_basic (clickhouse-client): new timeseries schema collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 schema = TimeseriesSchema { timeseries_name: TimeseriesName("collection_target:virtual_disk_space_provisioned"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "id", field_type: Uuid, source: Target, description: "" }}, datum_type: I64, version: 1, authz_scope: Fleet, units: Count, created: 2025-04-30T06:14:27.443803769Z } timeseries_name = collection_target:virtual_disk_space_provisioned
61902025-04-30T06:14:27.462ZDEBGtest_basic (clickhouse-client): inserting 4 new timeseries schema collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812
61912025-04-30T06:14:27.462ZTRACtest_basic (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_columns = 6 n_rows = 4 sql = INSERT INTO oximeter.timeseries_schema FORMAT Native
61922025-04-30T06:14:27.462ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ed7150da-7624-4ed1-8454-7520e194c7d8", "content-length": "734", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
61932025-04-30T06:14:27.463ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=0
61942025-04-30T06:14:27.469ZTRACtest_basic (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_columns = 4 n_rows = 10 sql = INSERT INTO oximeter.fields_string FORMAT Native
61952025-04-30T06:14:27.470ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a0ef5f33-4621-42ee-b0dd-5d768a36d9e5", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
61962025-04-30T06:14:27.470ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=1
61972025-04-30T06:14:27.477ZDEBGtest_basic (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_rows = 10 table_name = fields_string
61982025-04-30T06:14:27.477ZTRACtest_basic (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_columns = 4 n_rows = 5 sql = INSERT INTO oximeter.fields_u16 FORMAT Native
61992025-04-30T06:14:27.485ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "039ed137-7a14-4380-a109-aafcf3c91a65", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
62002025-04-30T06:14:27.485ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=0
62012025-04-30T06:14:27.492ZDEBGtest_basic (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_rows = 5 table_name = fields_u16
62022025-04-30T06:14:27.492ZTRACtest_basic (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_columns = 4 n_rows = 17 sql = INSERT INTO oximeter.fields_uuid FORMAT Native
62032025-04-30T06:14:27.518ZWARNtest_basic (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
62042025-04-30T06:14:27.524ZDEBGtest_basic (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_rows = 17 table_name = fields_uuid
62052025-04-30T06:14:27.525ZTRACtest_basic (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_columns = 19 n_rows = 5 sql = INSERT INTO oximeter.measurements_histogramu64 FORMAT Native
62062025-04-30T06:14:27.529ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ff0fada7-a38d-4b85-b28c-50ad1796d483", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
62072025-04-30T06:14:27.529ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=1
62082025-04-30T06:14:27.536ZDEBGtest_basic (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_rows = 5 table_name = measurements_histogramu64
62092025-04-30T06:14:27.537ZTRACtest_basic (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_columns = 4 n_rows = 12 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
62102025-04-30T06:14:27.543ZDEBGtest_basic (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_rows = 12 table_name = measurements_i64
62112025-04-30T06:14:27.543ZTRACtest_basic (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
62122025-04-30T06:14:27.544ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "af06cb6a-be26-4569-9a97-587b5518e80e", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
62132025-04-30T06:14:27.548ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=0
62142025-04-30T06:14:27.556ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cfa08763-bd03-46f6-9220-d1cf94588c73", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
62152025-04-30T06:14:27.558ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=1
62162025-04-30T06:14:27.571ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cb1ca33f-8a6b-441a-a43a-ec211990f3c6", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
62172025-04-30T06:14:27.572ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cmpa
62182025-04-30T06:14:27.573ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e04b8b73-48c6-4d42-9729-022661aea3ed", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
62192025-04-30T06:14:27.574ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
62202025-04-30T06:14:27.578ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e3d75b5a-959e-4f2d-8a0e-9a1037b39c1d", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
62212025-04-30T06:14:27.579ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
62222025-04-30T06:14:27.580ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8ce788b2-24a3-41b8-9121-f32341d1904e", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
62232025-04-30T06:14:27.581ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
62242025-04-30T06:14:27.582ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0b40136e-d52c-430b-bae4-dc988da4cd39", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
62252025-04-30T06:14:27.583ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1
62262025-04-30T06:14:27.585ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "dcaf2df9-2205-40ec-aa06-0c6a51646f1b", "content-length": "734", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
62272025-04-30T06:14:27.585ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=0
62282025-04-30T06:14:27.601ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d36444f3-f638-4009-bc31-75db14877c4e", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
62292025-04-30T06:14:27.601ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=1
62302025-04-30T06:14:27.606ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc2c9735-6027-4351-bf82-93b16cc40537", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
62312025-04-30T06:14:27.607ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=0
62322025-04-30T06:14:27.613ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "abbea7f6-6d51-4b87-afb0-1fdd87a6b029", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
62332025-04-30T06:14:27.614ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=1
62342025-04-30T06:14:27.617ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "97e4237f-7bd9-47aa-ad29-5cd146ea0e1e", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
62352025-04-30T06:14:27.617ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=0
62362025-04-30T06:14:27.628ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "83c7413b-c03e-4549-8760-24aa46e57b20", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
62372025-04-30T06:14:27.632ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=1
62382025-04-30T06:14:27.643ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "43ebe2d0-c844-4729-a838-f6d085b076e2", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
62392025-04-30T06:14:27.643ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cmpa
62402025-04-30T06:14:27.644ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7f5c473c-eb6d-4866-92fd-a2de4becf6db", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
62412025-04-30T06:14:27.645ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
62422025-04-30T06:14:27.647ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d6292743-b84c-40ba-8dec-334bc9ff25af", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
62432025-04-30T06:14:27.647ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
62442025-04-30T06:14:27.649ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "744618ff-3ef9-42c2-a1a0-8a51d4c365b7", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
62452025-04-30T06:14:27.649ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
62462025-04-30T06:14:27.652ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "35d5ea69-e2f2-4818-a95a-38784012d768", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
62472025-04-30T06:14:27.652ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0
62482025-04-30T06:14:27.656ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b8e78adb-ef42-44e8-9150-8b9d35bb91a9", "content-length": "723", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
62492025-04-30T06:14:27.657ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=0
62502025-04-30T06:14:27.661ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b4daa010-dd91-4f3a-a80e-6d1b62a41813", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
62512025-04-30T06:14:27.661ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=1
62522025-04-30T06:14:27.665ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ca791b30-3890-4eb4-a9f4-b463524d6204", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
62532025-04-30T06:14:27.665ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=0
62542025-04-30T06:14:27.667ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "954ce023-8231-4276-b49b-cfd49b88ae1f", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
62552025-04-30T06:14:27.667ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=1
62562025-04-30T06:14:27.669ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cf1d0da5-01a6-44ca-b00a-2bbe46c55dac", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
62572025-04-30T06:14:27.669ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=0
62582025-04-30T06:14:27.671ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3673c37b-9f4b-41ca-a9ae-f853cc752301", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
62592025-04-30T06:14:27.671ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=1
62602025-04-30T06:14:27.673ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9ff8c737-03d6-4e80-a82a-85e964b47b83", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
62612025-04-30T06:14:27.674ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cmpa
62622025-04-30T06:14:27.675ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "215539a3-333c-4efe-b0da-931d4c4b539e", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
62632025-04-30T06:14:27.675ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
62642025-04-30T06:14:27.676ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "434c7534-beae-48e8-bb70-126fdbee16b5", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
62652025-04-30T06:14:27.676ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
62662025-04-30T06:14:27.677ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e0a47844-c2fb-4aa3-9936-c74c1a8ee319", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
62672025-04-30T06:14:27.677ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
62682025-04-30T06:14:27.686ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "dec5108d-79f7-4f37-a0d1-c90a42038468", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
62692025-04-30T06:14:27.686ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1
62702025-04-30T06:14:27.688ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3e0aa7a0-898b-4bbf-be54-ec71ccc6f77a", "content-length": "723", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
62712025-04-30T06:14:27.688ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=0
62722025-04-30T06:14:27.691ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a21195c4-c800-4fc4-b741-5ba2670086f0", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
62732025-04-30T06:14:27.691ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=1
62742025-04-30T06:14:27.693ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b34a9fb0-7cfa-4f64-9591-3f616f006462", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
62752025-04-30T06:14:27.694ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=0
62762025-04-30T06:14:27.696ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "04882ef2-972d-4d01-8ec4-e51dafd8b4cc", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
62772025-04-30T06:14:27.696ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=1
62782025-04-30T06:14:27.698ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "735a3d29-2375-4595-8003-abccc71bc184", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
62792025-04-30T06:14:27.698ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=0
62802025-04-30T06:14:27.701ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "103971e5-e177-44c3-af39-f7c07b3f718c", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
62812025-04-30T06:14:27.701ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=1
62822025-04-30T06:14:27.703ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b3b86506-6ecb-4da1-bf58-4a28c7831fcb", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
62832025-04-30T06:14:27.703ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cmpa
62842025-04-30T06:14:27.703ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6edf32a1-23e0-44b5-9a33-2538e4b87306", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
62852025-04-30T06:14:27.704ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
62862025-04-30T06:14:27.705ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3c00bfe5-aac5-4170-8417-a16e434dddd8", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
62872025-04-30T06:14:27.705ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
62882025-04-30T06:14:27.706ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "dcb7b255-f36f-4bed-a143-02c90ab7131d", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
62892025-04-30T06:14:27.706ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
62902025-04-30T06:14:27.707ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7e047b80-0128-41a9-9ba4-fbd63b53824d", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
62912025-04-30T06:14:27.707ZTRACtest_basic: authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet
62922025-04-30T06:14:27.707ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
62932025-04-30T06:14:27.707ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
62942025-04-30T06:14:27.707ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
62952025-04-30T06:14:27.708ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
62962025-04-30T06:14:27.718ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62972025-04-30T06:14:27.728ZDEBGtest_basic: authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet result = Ok(())
62982025-04-30T06:14:27.728ZTRACtest_basic: authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet
62992025-04-30T06:14:27.728ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
63002025-04-30T06:14:27.728ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
63012025-04-30T06:14:27.728ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
63022025-04-30T06:14:27.729ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
63032025-04-30T06:14:27.730ZDEBGtest_basic (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
63042025-04-30T06:14:27.730ZDEBGtest_basic (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
63052025-04-30T06:14:27.731ZDEBGtest_basic (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812
63062025-04-30T06:14:27.735ZTRACtest_basic (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
63072025-04-30T06:14:27.747ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63082025-04-30T06:14:27.783ZDEBGtest_basic: authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet result = Ok(())
63092025-04-30T06:14:27.784ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
63102025-04-30T06:14:27.784ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
63112025-04-30T06:14:27.788ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
63122025-04-30T06:14:27.885ZDEBGtest_basic: begin collection from Sled Agent sled_agent_url = http://[::1]:42420
63132025-04-30T06:14:27.885ZDEBGtest_basic: client request SledAgent = http://[::1]:42420 body = None method = GET uri = http://[::1]:42420/inventory
63142025-04-30T06:14:27.886ZINFOtest_basic (dropshot): accepted connection local_addr = [::1]:42420 remote_addr = [::1]:48755 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
63152025-04-30T06:14:27.886ZDEBGtest_basic (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_rows = 2 table_name = fields_string
63162025-04-30T06:14:27.887ZTRACtest_basic (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
63172025-04-30T06:14:27.888ZWARNtest_basic (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
63182025-04-30T06:14:27.888ZTRACtest_basic (dropshot): incoming request local_addr = [::1]:42420 method = GET remote_addr = [::1]:48755 req_id = 5a50f21a-5d61-41c5-871c-1cf6b7d662be sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
63192025-04-30T06:14:27.890ZINFOtest_basic (dropshot): request completed latency_us = 1325 local_addr = [::1]:42420 method = GET remote_addr = [::1]:48755 req_id = 5a50f21a-5d61-41c5-871c-1cf6b7d662be response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
63202025-04-30T06:14:27.891ZDEBGtest_basic: client response SledAgent = http://[::1]:42420 result = Ok(Response { url: "http://[::1]:42420/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "5a50f21a-5d61-41c5-871c-1cf6b7d662be", "content-length": "2769", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
63212025-04-30T06:14:27.895ZDEBGtest_basic: begin collecting all keepers nkeeper_admin_clients = 0
63222025-04-30T06:14:27.895ZDEBGtest_basic: end collecting all keepers nkeeper_admin_clients = 0
63232025-04-30T06:14:27.895ZDEBGtest_basic: finished collection
63242025-04-30T06:14:27.898ZTRACtest_basic: authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory
63252025-04-30T06:14:27.898ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
63262025-04-30T06:14:27.898ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
63272025-04-30T06:14:27.899ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
63282025-04-30T06:14:27.899ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
63292025-04-30T06:14:27.900ZDEBGtest_basic (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_rows = 1 table_name = measurements_i64
63302025-04-30T06:14:27.900ZTRACtest_basic (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
63312025-04-30T06:14:27.903ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63322025-04-30T06:14:27.910ZDEBGtest_basic: authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory result = Ok(())
63332025-04-30T06:14:27.910ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
63342025-04-30T06:14:27.910ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
63352025-04-30T06:14:27.911ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
63362025-04-30T06:14:27.960ZWARNtest_basic (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
63372025-04-30T06:14:27.961ZWARNtest_basic: inventory collection failed error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
63382025-04-30T06:14:28.005ZTRACtest_basic: authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory
63392025-04-30T06:14:28.005ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
63402025-04-30T06:14:28.005ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
63412025-04-30T06:14:28.007ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
63422025-04-30T06:14:28.012ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
63432025-04-30T06:14:28.025ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63442025-04-30T06:14:28.031ZDEBGtest_basic: authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory result = Ok(())
63452025-04-30T06:14:28.031ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
63462025-04-30T06:14:28.032ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
63472025-04-30T06:14:28.032ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
63482025-04-30T06:14:28.038ZDEBGtest_basic: inventory_prune_one: nothing eligible for removal (too few) candidates = [(efcb0ee5-cafa-4692-9b75-fcee4732bdca, 0)]
63492025-04-30T06:14:28.044ZTRACtest_basic: lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
63502025-04-30T06:14:28.045ZTRACtest_basic (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 10 01 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:51068 req_id = c5fa9ce3-3218-4edf-8741-ec329460ce4d
63512025-04-30T06:14:28.045ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 4097,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51068 req_id = c5fa9ce3-3218-4edf-8741-ec329460ce4d
63522025-04-30T06:14:28.047ZDEBGtest_basic (store): query key key = _mgs._tcp
63532025-04-30T06:14:28.047ZDEBGtest_basic (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
63542025-04-30T06:14:28.047ZDEBGtest_basic (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:51068 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 37822, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = c5fa9ce3-3218-4edf-8741-ec329460ce4d
63552025-04-30T06:14:28.048ZDEBGtest_basic: lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 37822, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2532, tv_nsec: 846952202 } })
63562025-04-30T06:14:28.048ZTRACtest_basic: lookup_service_targets: looking up SRV target
63572025-04-30T06:14:28.048ZTRACtest_basic (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 36 37 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e 67..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:54934 req_id = 117e96a8-cdc9-47ad-9cea-472a60e36836
63582025-04-30T06:14:28.049ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 13879,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54934 req_id = 117e96a8-cdc9-47ad-9cea-472a60e36836
63592025-04-30T06:14:28.049ZDEBGtest_basic (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
63602025-04-30T06:14:28.049ZDEBGtest_basic (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:54934 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 117e96a8-cdc9-47ad-9cea-472a60e36836
63612025-04-30T06:14:28.118ZTRACtest_basic: lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
63622025-04-30T06:14:28.121ZTRACtest_basic (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 37 6c 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c 7l..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:62020 req_id = 88f419f2-5615-4df7-b4fd-b693900058b4
63632025-04-30T06:14:28.121ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 14188,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62020 req_id = 88f419f2-5615-4df7-b4fd-b693900058b4
63642025-04-30T06:14:28.121ZDEBGtest_basic (store): query key key = _clickhouse-admin-keeper._tcp
63652025-04-30T06:14:28.121ZERROtest_basic (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 14188, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:62020 req_id = 88f419f2-5615-4df7-b4fd-b693900058b4
63662025-04-30T06:14:28.121ZDEBGtest_basic: begin collection
63672025-04-30T06:14:28.121ZDEBGtest_basic: begin collection from MGS mgs_url = http://[::1]:37822
63682025-04-30T06:14:28.121ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/ignition
63692025-04-30T06:14:28.139ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "d1adc202-4e2c-477c-99c0-4e4fb552cd48", "content-length": "835", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
63702025-04-30T06:14:28.140ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0
63712025-04-30T06:14:28.161ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "25f5e2a2-a0ce-4dcc-96f2-3fe1354384e9", "content-length": "734", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
63722025-04-30T06:14:28.161ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=0
63732025-04-30T06:14:28.167ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f1d3cc3d-4d0d-4b9f-91ca-caeadf7812b1", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
63742025-04-30T06:14:28.167ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=1
63752025-04-30T06:14:28.171ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d0fe3b22-308e-4ecd-9b83-28ba691785c1", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
63762025-04-30T06:14:28.172ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=0
63772025-04-30T06:14:28.174ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cb61b376-cff0-4669-9f8d-34d93ee4aa88", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
63782025-04-30T06:14:28.174ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=1
63792025-04-30T06:14:28.176ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4187bcff-fb8b-4d0c-b786-efda4ce895c9", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
63802025-04-30T06:14:28.177ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=0
63812025-04-30T06:14:28.182ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b5df2424-3b60-483c-a366-60e12cc81ccc", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
63822025-04-30T06:14:28.182ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=1
63832025-04-30T06:14:28.185ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fbbba45d-6f5c-4886-9ea4-296e72ccdeaf", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
63842025-04-30T06:14:28.185ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cmpa
63852025-04-30T06:14:28.187ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b043a720-a056-4522-806d-e431fa607b2f", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
63862025-04-30T06:14:28.187ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
63872025-04-30T06:14:28.188ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8076f5f2-ab5d-4ff3-a140-421d4ef72793", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
63882025-04-30T06:14:28.189ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
63892025-04-30T06:14:28.190ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0b64d1a6-04c8-4e0a-b24e-b01e3a3b3668", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
63902025-04-30T06:14:28.190ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
63912025-04-30T06:14:28.194ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3e926caf-f332-41d4-a86d-3ac17458390c", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
63922025-04-30T06:14:28.194ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1
63932025-04-30T06:14:28.199ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b037e58b-77e7-4357-9b98-aba1dd21ec62", "content-length": "734", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
63942025-04-30T06:14:28.201ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=0
63952025-04-30T06:14:28.204ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "59b3933a-bddb-4d39-a4ac-f6c23b71e74f", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
63962025-04-30T06:14:28.204ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=1
63972025-04-30T06:14:28.208ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cda80c50-0d3d-4a02-b096-712bf9bf01ef", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
63982025-04-30T06:14:28.208ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=0
63992025-04-30T06:14:28.212ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1857331c-f13f-4288-bbbb-5210f39cc58f", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
64002025-04-30T06:14:28.212ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=1
64012025-04-30T06:14:28.216ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "43f08361-0b98-4b02-a93d-69b64b8c302e", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
64022025-04-30T06:14:28.216ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=0
64032025-04-30T06:14:28.218ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8f352614-9c4c-4236-9f06-63525a14ef8b", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
64042025-04-30T06:14:28.218ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=1
64052025-04-30T06:14:28.223ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f42e0748-c16d-4a3b-9803-df93d98617b4", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
64062025-04-30T06:14:28.223ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cmpa
64072025-04-30T06:14:28.225ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "78b92d3d-cbc6-4df2-8437-644c16c0e850", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
64082025-04-30T06:14:28.225ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
64092025-04-30T06:14:28.226ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c5757381-6824-4b12-ade5-89edc9215b05", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
64102025-04-30T06:14:28.227ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
64112025-04-30T06:14:28.230ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8847298b-fc93-4862-bba6-5ba71d35b345", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
64122025-04-30T06:14:28.230ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
64132025-04-30T06:14:28.233ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f2403f36-28a2-4f2a-90b3-2873cb57d978", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
64142025-04-30T06:14:28.233ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0
64152025-04-30T06:14:28.235ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d083e608-b616-452e-9751-717a44d3a7e1", "content-length": "723", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
64162025-04-30T06:14:28.237ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=0
64172025-04-30T06:14:28.242ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5c0714c3-0e76-4760-9385-93ed55ed056f", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
64182025-04-30T06:14:28.244ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=1
64192025-04-30T06:14:28.248ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e96ebccb-0f99-461a-a450-6c165d78ce0e", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
64202025-04-30T06:14:28.249ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=0
64212025-04-30T06:14:28.256ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9b1c1ca6-d2e6-4c05-987b-a26a0501c599", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
64222025-04-30T06:14:28.256ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=1
64232025-04-30T06:14:28.269ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "46d31726-a61e-4017-8cc1-daf9a8217ffe", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
64242025-04-30T06:14:28.270ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=0
64252025-04-30T06:14:28.274ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "97b199dc-18c4-4462-8f87-aee40ee120e3", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
64262025-04-30T06:14:28.274ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=1
64272025-04-30T06:14:28.281ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "dfa4905c-c8ac-4e9b-a5de-382faeee1b43", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
64282025-04-30T06:14:28.281ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cmpa
64292025-04-30T06:14:28.283ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6934094f-9f9d-40e4-a167-6e4b514a4214", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
64302025-04-30T06:14:28.283ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
64312025-04-30T06:14:28.285ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "22a31f48-29ac-48d6-9a75-3f3f75309491", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
64322025-04-30T06:14:28.285ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
64332025-04-30T06:14:28.287ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "978295f0-7dd6-41ac-a426-122b5fe7bfca", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
64342025-04-30T06:14:28.287ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
64352025-04-30T06:14:28.289ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ccc06602-784f-4b0d-a4d8-63bc414a5d9b", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
64362025-04-30T06:14:28.290ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1
64372025-04-30T06:14:28.292ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9f6f2b9f-5e53-406a-af5f-27a2f8b31ed0", "content-length": "723", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
64382025-04-30T06:14:28.293ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=0
64392025-04-30T06:14:28.309ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "adc1281d-338d-4326-92e8-5ed8742c9eb8", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
64402025-04-30T06:14:28.310ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=1
64412025-04-30T06:14:28.319ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e5fbb48e-de8a-4265-b3eb-318d2a4e35dd", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
64422025-04-30T06:14:28.319ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=0
64432025-04-30T06:14:28.322ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c65044e2-0fa8-4eca-8609-bed022081baf", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
64442025-04-30T06:14:28.322ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=1
64452025-04-30T06:14:28.324ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ca90e420-5e7d-4564-aea0-d13f425c5e52", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:27 GMT"} })
64462025-04-30T06:14:28.324ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=0
64472025-04-30T06:14:28.326ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ab173fd2-e4e2-4e85-85fb-e00ab8530f06", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
64482025-04-30T06:14:28.326ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=1
64492025-04-30T06:14:28.329ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a8f16658-09dc-461d-8301-7f3f2ced6213", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
64502025-04-30T06:14:28.330ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cmpa
64512025-04-30T06:14:28.330ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9569fdce-c5e0-405b-8be0-edc5e2da86bf", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
64522025-04-30T06:14:28.330ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
64532025-04-30T06:14:28.331ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7d964d6e-4527-477c-8dda-330af1821446", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
64542025-04-30T06:14:28.332ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
64552025-04-30T06:14:28.333ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c738fb25-fd0b-4e3c-ae58-b30b01d0387e", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
64562025-04-30T06:14:28.333ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
64572025-04-30T06:14:28.334ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5e2d9068-1c6d-4d05-bab6-b89289230315", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
64582025-04-30T06:14:28.335ZTRACtest_basic: authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet
64592025-04-30T06:14:28.335ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
64602025-04-30T06:14:28.335ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
64612025-04-30T06:14:28.335ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
64622025-04-30T06:14:28.337ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
64632025-04-30T06:14:28.341ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64642025-04-30T06:14:28.349ZDEBGtest_basic: authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet result = Ok(())
64652025-04-30T06:14:28.349ZTRACtest_basic: authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet
64662025-04-30T06:14:28.349ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
64672025-04-30T06:14:28.349ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
64682025-04-30T06:14:28.350ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
64692025-04-30T06:14:28.353ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
64702025-04-30T06:14:28.356ZDEBGtest_basic (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
64712025-04-30T06:14:28.356ZDEBGtest_basic (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
64722025-04-30T06:14:28.358ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64732025-04-30T06:14:28.378ZDEBGtest_basic: authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet result = Ok(())
64742025-04-30T06:14:28.378ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
64752025-04-30T06:14:28.378ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
64762025-04-30T06:14:28.379ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
64772025-04-30T06:14:28.382ZTRACtest_basic (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
64782025-04-30T06:14:28.383ZDEBGtest_basic (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
64792025-04-30T06:14:28.383ZTRACtest_basic (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
64802025-04-30T06:14:28.383ZTRACtest_basic (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
64812025-04-30T06:14:28.460ZDEBGtest_basic: begin collection from Sled Agent sled_agent_url = http://[::1]:42420
64822025-04-30T06:14:28.460ZDEBGtest_basic: client request SledAgent = http://[::1]:42420 body = None method = GET uri = http://[::1]:42420/inventory
64832025-04-30T06:14:28.461ZINFOtest_basic (dropshot): accepted connection local_addr = [::1]:42420 remote_addr = [::1]:42056 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
64842025-04-30T06:14:28.461ZTRACtest_basic (dropshot): incoming request local_addr = [::1]:42420 method = GET remote_addr = [::1]:42056 req_id = 98ca369e-d95f-4282-8464-4047e7da70ea sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
64852025-04-30T06:14:28.462ZINFOtest_basic (dropshot): request completed latency_us = 596 local_addr = [::1]:42420 method = GET remote_addr = [::1]:42056 req_id = 98ca369e-d95f-4282-8464-4047e7da70ea response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
64862025-04-30T06:14:28.462ZDEBGtest_basic: client response SledAgent = http://[::1]:42420 result = Ok(Response { url: "http://[::1]:42420/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "98ca369e-d95f-4282-8464-4047e7da70ea", "content-length": "2769", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
64872025-04-30T06:14:28.465ZDEBGtest_basic: begin collecting all keepers nkeeper_admin_clients = 0
64882025-04-30T06:14:28.465ZDEBGtest_basic: end collecting all keepers nkeeper_admin_clients = 0
64892025-04-30T06:14:28.465ZDEBGtest_basic: finished collection
64902025-04-30T06:14:28.468ZTRACtest_basic: authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory
64912025-04-30T06:14:28.468ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
64922025-04-30T06:14:28.469ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
64932025-04-30T06:14:28.469ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
64942025-04-30T06:14:28.470ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
64952025-04-30T06:14:28.474ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64962025-04-30T06:14:28.482ZDEBGtest_basic: authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory result = Ok(())
64972025-04-30T06:14:28.483ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
64982025-04-30T06:14:28.483ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
64992025-04-30T06:14:28.483ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
65002025-04-30T06:14:28.525ZWARNtest_basic (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
65012025-04-30T06:14:28.528ZWARNtest_basic: inventory collection failed error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
65022025-04-30T06:14:28.537ZTRACtest_basic: authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory
65032025-04-30T06:14:28.537ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
65042025-04-30T06:14:28.537ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
65052025-04-30T06:14:28.537ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
65062025-04-30T06:14:28.538ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
65072025-04-30T06:14:28.543ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65082025-04-30T06:14:28.554ZDEBGtest_basic: authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory result = Ok(())
65092025-04-30T06:14:28.554ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
65102025-04-30T06:14:28.554ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
65112025-04-30T06:14:28.555ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
65122025-04-30T06:14:28.559ZDEBGtest_basic: inventory_prune_one: nothing eligible for removal (too few) candidates = [(efcb0ee5-cafa-4692-9b75-fcee4732bdca, 0)]
65132025-04-30T06:14:28.559ZTRACtest_basic: lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
65142025-04-30T06:14:28.561ZTRACtest_basic (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 86 36 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .6..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:52822 req_id = d1f6e989-3a17-4b20-8c71-6c62e592e9cd
65152025-04-30T06:14:28.561ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 34358,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52822 req_id = d1f6e989-3a17-4b20-8c71-6c62e592e9cd
65162025-04-30T06:14:28.561ZDEBGtest_basic (store): query key key = _mgs._tcp
65172025-04-30T06:14:28.561ZDEBGtest_basic (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
65182025-04-30T06:14:28.562ZDEBGtest_basic (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:52822 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 37822, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = d1f6e989-3a17-4b20-8c71-6c62e592e9cd
65192025-04-30T06:14:28.563ZDEBGtest_basic: lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 37822, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2533, tv_nsec: 361928629 } })
65202025-04-30T06:14:28.563ZTRACtest_basic: lookup_service_targets: looking up SRV target
65212025-04-30T06:14:28.563ZTRACtest_basic (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 22 05 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e "...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:50283 req_id = 337cf009-6628-472e-8ada-41bf747ec053
65222025-04-30T06:14:28.563ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 8709,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:50283 req_id = 337cf009-6628-472e-8ada-41bf747ec053
65232025-04-30T06:14:28.563ZDEBGtest_basic (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
65242025-04-30T06:14:28.564ZDEBGtest_basic (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:50283 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 337cf009-6628-472e-8ada-41bf747ec053
65252025-04-30T06:14:28.623ZTRACtest_basic: lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
65262025-04-30T06:14:28.623ZTRACtest_basic (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 07 9c 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:62258 req_id = abde6fe5-07c8-4b4f-bea3-de28924a05d4
65272025-04-30T06:14:28.624ZDEBGtest_basic (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 1948,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62258 req_id = abde6fe5-07c8-4b4f-bea3-de28924a05d4
65282025-04-30T06:14:28.624ZDEBGtest_basic (store): query key key = _clickhouse-admin-keeper._tcp
65292025-04-30T06:14:28.624ZERROtest_basic (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 1948, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:62258 req_id = abde6fe5-07c8-4b4f-bea3-de28924a05d4
65302025-04-30T06:14:28.625ZDEBGtest_basic: begin collection
65312025-04-30T06:14:28.625ZDEBGtest_basic: begin collection from MGS mgs_url = http://[::1]:37822
65322025-04-30T06:14:28.626ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/ignition
65332025-04-30T06:14:28.628ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "c29e1725-9836-4183-9f75-f8b47f3d02e0", "content-length": "835", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
65342025-04-30T06:14:28.629ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0
65352025-04-30T06:14:28.630ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f6527631-f701-469e-8a67-914422e5fb09", "content-length": "734", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
65362025-04-30T06:14:28.630ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=0
65372025-04-30T06:14:28.636ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9a6f8804-7512-4ee1-ac76-23119b815d65", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
65382025-04-30T06:14:28.636ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=1
65392025-04-30T06:14:28.638ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d74c2583-2c49-484b-87d2-858db9a911f6", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
65402025-04-30T06:14:28.638ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=0
65412025-04-30T06:14:28.641ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "03b259a3-4815-49ee-bb8e-84c8f2e2ec49", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
65422025-04-30T06:14:28.641ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=1
65432025-04-30T06:14:28.642ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "545d7f6e-24af-4f7c-b324-e4dab5d694d2", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
65442025-04-30T06:14:28.643ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=0
65452025-04-30T06:14:28.644ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e8271a37-55b7-41e7-ba84-f28395fd4166", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
65462025-04-30T06:14:28.644ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=1
65472025-04-30T06:14:28.646ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b9f6ae42-7178-4e5c-b5b5-00a999fdaba5", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
65482025-04-30T06:14:28.646ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cmpa
65492025-04-30T06:14:28.648ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7a9f5cfc-234e-4dce-8d4d-264d50be0ee7", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
65502025-04-30T06:14:28.648ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
65512025-04-30T06:14:28.649ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9ff0b2fd-6003-4ced-a0c7-e4daa725bf6c", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
65522025-04-30T06:14:28.649ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
65532025-04-30T06:14:28.651ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5dc66dcf-ad03-49b3-a821-afc338454aaf", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
65542025-04-30T06:14:28.651ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/0/component/rot/cfpa
65552025-04-30T06:14:28.653ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e3632337-c83b-4ce9-b88f-c7174f7ced42", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
65562025-04-30T06:14:28.653ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1
65572025-04-30T06:14:28.656ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "214dffed-3381-4bc0-b96b-bb0bade4efd5", "content-length": "734", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
65582025-04-30T06:14:28.656ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=0
65592025-04-30T06:14:28.659ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8b4b639b-ec39-4f0f-97a6-50dd2e69c89e", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
65602025-04-30T06:14:28.659ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=1
65612025-04-30T06:14:28.662ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "41184393-8277-42eb-8bc6-2d96206c7564", "content-length": "111", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
65622025-04-30T06:14:28.662ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=0
65632025-04-30T06:14:28.665ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ebb85601-ff69-495a-b54a-647fb0cc3583", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
65642025-04-30T06:14:28.665ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=1
65652025-04-30T06:14:28.668ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "633f6766-4c77-408e-836d-becc6e625839", "content-length": "170", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
65662025-04-30T06:14:28.669ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=0
65672025-04-30T06:14:28.673ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0085d7f7-e9fc-47e5-b9b1-f2a265c9feca", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
65682025-04-30T06:14:28.673ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=1
65692025-04-30T06:14:28.675ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "74c3a9fa-a37f-4cb9-a023-cf34fddd188a", "content-length": "179", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
65702025-04-30T06:14:28.675ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cmpa
65712025-04-30T06:14:28.676ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d591fd4f-923d-4837-b997-d4c18d95e26d", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
65722025-04-30T06:14:28.677ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
65732025-04-30T06:14:28.679ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "98607a00-9d90-4128-92e0-df05eab4e391", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
65742025-04-30T06:14:28.679ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
65752025-04-30T06:14:28.681ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "950ec3d8-0a17-4d6f-a19e-74618183a4dd", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
65762025-04-30T06:14:28.681ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/switch/1/component/rot/cfpa
65772025-04-30T06:14:28.682ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "eb17719c-eccb-45e9-93d4-d374428b272b", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
65782025-04-30T06:14:28.684ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0
65792025-04-30T06:14:28.685ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7ed6ee23-9ded-44fb-9323-e613efcbc407", "content-length": "723", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
65802025-04-30T06:14:28.686ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=0
65812025-04-30T06:14:28.690ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "23e58186-b454-48a8-930e-2b581bace065", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
65822025-04-30T06:14:28.690ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=1
65832025-04-30T06:14:28.693ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d7bbc2ec-2999-4ec8-9871-76ccd67cf56f", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
65842025-04-30T06:14:28.694ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=0
65852025-04-30T06:14:28.696ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "19c1d219-9bb3-40c8-9477-6a2c7fbb9977", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
65862025-04-30T06:14:28.696ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=1
65872025-04-30T06:14:28.700ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "afca81c9-f787-4f18-a8da-5fe7cc0eee38", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
65882025-04-30T06:14:28.700ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=0
65892025-04-30T06:14:28.707ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b338b3c3-5403-40f9-b860-ba1e8d9beae3", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
65902025-04-30T06:14:28.707ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=1
65912025-04-30T06:14:28.715ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd7ff522-734a-4582-836d-0aeba6b39c22", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
65922025-04-30T06:14:28.715ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cmpa
65932025-04-30T06:14:28.720ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "18b4d36f-c04f-4104-b0b6-e26093e0a085", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
65942025-04-30T06:14:28.723ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
65952025-04-30T06:14:28.724ZDEBGtest_basic (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
65962025-04-30T06:14:28.724ZDEBGtest_basic (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
65972025-04-30T06:14:28.725ZDEBGtest_basic (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812
65982025-04-30T06:14:28.727ZTRACtest_basic (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
65992025-04-30T06:14:28.728ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "11e6027e-d7bb-4db3-9c97-308d6a8eed27", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
66002025-04-30T06:14:28.728ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
66012025-04-30T06:14:28.735ZDEBGtest_basic (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_rows = 2 table_name = fields_string
66022025-04-30T06:14:28.735ZTRACtest_basic (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
66032025-04-30T06:14:28.740ZDEBGtest_basic (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 852e9b0a-c261-4054-802e-b1d874643812 n_rows = 1 table_name = measurements_i64
66042025-04-30T06:14:28.740ZTRACtest_basic (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
66052025-04-30T06:14:28.740ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d0ae3fc9-0188-42db-9cfb-95d1aa6c9816", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
66062025-04-30T06:14:28.741ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/0/component/rot/cfpa
66072025-04-30T06:14:28.743ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "22092f9e-8e7c-4154-96b4-33d59964648b", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
66082025-04-30T06:14:28.751ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1
66092025-04-30T06:14:28.759ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "64f313c8-532d-478b-b716-5b828e7a3dfc", "content-length": "723", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
66102025-04-30T06:14:28.759ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=0
66112025-04-30T06:14:28.767ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "527c2225-a191-46fe-95a9-fc59a2787e4c", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
66122025-04-30T06:14:28.768ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=1
66132025-04-30T06:14:28.778ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0e7b2fb5-1d74-4c51-bfc5-c2f13460e474", "content-length": "109", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
66142025-04-30T06:14:28.779ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=0
66152025-04-30T06:14:28.788ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "892c420f-22af-4664-ab3c-3780fbe20927", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
66162025-04-30T06:14:28.788ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=1
66172025-04-30T06:14:28.792ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c7d7f8fe-0d40-4643-920a-3009b24a7c1b", "content-length": "169", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
66182025-04-30T06:14:28.793ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=0
66192025-04-30T06:14:28.801ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b7791842-92d5-4347-93ee-23b4938d8cbe", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
66202025-04-30T06:14:28.802ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=1
66212025-04-30T06:14:28.817ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "61205a9e-ecdb-4a5d-a0ca-6d855663faee", "content-length": "178", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
66222025-04-30T06:14:28.817ZDEBGtest_basic: client request body = None gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cmpa
66232025-04-30T06:14:28.820ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b9480c09-c869-47b2-96b1-826ecfabbfd4", "content-length": "702", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
66242025-04-30T06:14:28.822ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
66252025-04-30T06:14:28.841ZWARNtest_basic (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
66262025-04-30T06:14:28.842ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c1295007-5cd3-42d3-956e-a43210deb2f1", "content-length": "727", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
66272025-04-30T06:14:28.845ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
66282025-04-30T06:14:28.846ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "835b0931-d442-47a2-a808-40f1de67a95a", "content-length": "729", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
66292025-04-30T06:14:28.846ZDEBGtest_basic: client request body = Some(Body) gateway_url = http://[::1]:37822 method = GET uri = http://[::1]:37822/sp/sled/1/component/rot/cfpa
66302025-04-30T06:14:28.848ZDEBGtest_basic: client response gateway_url = http://[::1]:37822 result = Ok(Response { url: "http://[::1]:37822/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0cf69abd-338c-4835-86dd-c10a0a4e28a7", "content-length": "728", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
66312025-04-30T06:14:28.848ZTRACtest_basic: authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet
66322025-04-30T06:14:28.848ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
66332025-04-30T06:14:28.848ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
66342025-04-30T06:14:28.848ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
66352025-04-30T06:14:28.850ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
66362025-04-30T06:14:28.868ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66372025-04-30T06:14:28.872ZDEBGtest_basic: authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet result = Ok(())
66382025-04-30T06:14:28.872ZTRACtest_basic: authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet
66392025-04-30T06:14:28.872ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
66402025-04-30T06:14:28.872ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
66412025-04-30T06:14:28.872ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
66422025-04-30T06:14:28.873ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
66432025-04-30T06:14:28.878ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66442025-04-30T06:14:28.891ZDEBGtest_basic: authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Fleet result = Ok(())
66452025-04-30T06:14:28.891ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
66462025-04-30T06:14:28.891ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
66472025-04-30T06:14:28.892ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
66482025-04-30T06:14:28.966ZDEBGtest_basic: begin collection from Sled Agent sled_agent_url = http://[::1]:42420
66492025-04-30T06:14:28.966ZDEBGtest_basic: client request SledAgent = http://[::1]:42420 body = None method = GET uri = http://[::1]:42420/inventory
66502025-04-30T06:14:28.968ZINFOtest_basic (dropshot): accepted connection local_addr = [::1]:42420 remote_addr = [::1]:34639 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
66512025-04-30T06:14:28.968ZTRACtest_basic (dropshot): incoming request local_addr = [::1]:42420 method = GET remote_addr = [::1]:34639 req_id = 22d5fc7d-f948-4b98-9eba-c76fead9934b sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
66522025-04-30T06:14:28.969ZINFOtest_basic (dropshot): request completed latency_us = 492 local_addr = [::1]:42420 method = GET remote_addr = [::1]:34639 req_id = 22d5fc7d-f948-4b98-9eba-c76fead9934b response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
66532025-04-30T06:14:28.969ZDEBGtest_basic: client response SledAgent = http://[::1]:42420 result = Ok(Response { url: "http://[::1]:42420/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "22d5fc7d-f948-4b98-9eba-c76fead9934b", "content-length": "2769", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
66542025-04-30T06:14:28.972ZDEBGtest_basic: begin collecting all keepers nkeeper_admin_clients = 0
66552025-04-30T06:14:28.972ZDEBGtest_basic: end collecting all keepers nkeeper_admin_clients = 0
66562025-04-30T06:14:28.972ZDEBGtest_basic: finished collection
66572025-04-30T06:14:28.974ZTRACtest_basic: authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory
66582025-04-30T06:14:28.974ZTRACtest_basic: loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
66592025-04-30T06:14:28.974ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
66602025-04-30T06:14:28.974ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
66612025-04-30T06:14:28.976ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
66622025-04-30T06:14:28.982ZINFOtest_basic (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
66632025-04-30T06:14:28.982ZDEBGtest_basic (oximeter): client request body = None method = GET uri = http://127.0.0.1:34211/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
66642025-04-30T06:14:28.983ZTRACtest_basic (dropshot_internal): incoming request local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 41a8c3b6-0b67-4517-a12b-6b847b39d2aa uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
66652025-04-30T06:14:28.983ZTRACtest_basic (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 41a8c3b6-0b67-4517-a12b-6b847b39d2aa resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
66662025-04-30T06:14:28.983ZDEBGtest_basic (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 41a8c3b6-0b67-4517-a12b-6b847b39d2aa roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
66672025-04-30T06:14:28.984ZDEBGtest_basic (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 41a8c3b6-0b67-4517-a12b-6b847b39d2aa resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
66682025-04-30T06:14:28.985ZDEBGtest_basic: roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66692025-04-30T06:14:28.990ZDEBGtest_basic: authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Inventory result = Ok(())
66702025-04-30T06:14:28.991ZTRACtest_basic: authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database
66712025-04-30T06:14:28.991ZDEBGtest_basic: roles roles = RoleSet { roles: {} }
66722025-04-30T06:14:28.991ZDEBGtest_basic: authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) resource = Database result = Ok(())
66732025-04-30T06:14:28.992ZINFOtest_basic (dropshot_internal): request completed latency_us = 9325 local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 41a8c3b6-0b67-4517-a12b-6b847b39d2aa response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
66742025-04-30T06:14:28.996ZDEBGtest_basic (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:34211/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "41a8c3b6-0b67-4517-a12b-6b847b39d2aa", "content-length": "405", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
66752025-04-30T06:14:28.997ZDEBGtest_basic (oximeter): client request body = None method = GET uri = http://127.0.0.1:34211/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
66762025-04-30T06:14:28.998ZTRACtest_basic (dropshot_internal): incoming request local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 856d4447-9626-452f-a767-0f520dc566a5 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
66772025-04-30T06:14:29.004ZTRACtest_basic (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 856d4447-9626-452f-a767-0f520dc566a5 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
66782025-04-30T06:14:29.006ZDEBGtest_basic (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 856d4447-9626-452f-a767-0f520dc566a5 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
66792025-04-30T06:14:29.007ZDEBGtest_basic (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 856d4447-9626-452f-a767-0f520dc566a5 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
66802025-04-30T06:14:29.016ZINFOtest_basic (dropshot_internal): request completed latency_us = 17819 local_addr = 127.0.0.1:34211 method = GET remote_addr = 127.0.0.1:35990 req_id = 856d4447-9626-452f-a767-0f520dc566a5 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
66812025-04-30T06:14:29.020ZDEBGtest_basic (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:34211/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "856d4447-9626-452f-a767-0f520dc566a5", "content-length": "29", "date": "Wed, 30 Apr 2025 06:14:28 GMT"} })
66822025-04-30T06:14:29.022ZTRACtest_basic (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 088ac96c-981b-4054-bea6-d8ab23ae1366
66832025-04-30T06:14:29.023ZTRACtest_basic (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
66842025-04-30T06:14:29.023ZINFOtest_basic (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
66852025-04-30T06:14:29.107ZWARNtest_basic (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
66862025-04-30T06:14:29.107ZWARNtest_basic: inventory collection failed error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
66872025-04-30T06:14:29.128ZWARNtest_basic: inventory collection failed error = failed to collect inventory: disabled by explicit configuration