Max level shown:
12024-10-17T22:41:43.586ZDEBGtest_metrics (init_with_steps): Running step start_crdb
22024-10-17T22:41:43.586ZDEBGtest_metrics: Starting CRDB
32024-10-17T22:41:43.587ZINFOtest_metrics: cockroach temporary directory: /var/tmp/omicron_tmp/.tmpT7jBJJ
42024-10-17T22:41:43.587ZINFOtest_metrics: cockroach: copying from seed tarball (/var/tmp/omicron_tmp/crdb-base-build/715a51eb919177fa8d827f74e9ac217af9dd57c2e799d7d9e860efa5dba3a56f.tar) to storage directory (/var/tmp/omicron_tmp/.tmpT7jBJJ/data)
52024-10-17T22:41:43.625ZINFOtest_metrics: cockroach command line: cockroach start-single-node --insecure --http-addr=:0 --store=path=/var/tmp/omicron_tmp/.tmpT7jBJJ/data,ballast-size=0 --listen-addr [::1]:0 --listening-url-file /var/tmp/omicron_tmp/.tmpT7jBJJ/listen-url
62024-10-17T22:41:43.625ZINFOtest_metrics: cockroach environment: BUILDOMAT_JOB_ID=01JAE8A53QXBC84BXPH6CCK46C BUILDOMAT_TASK_ID=4 CARGO=/home/build/.rustup/toolchains/1.80.1-x86_64-unknown-linux-gnu/bin/cargo CARGO_HOME=/home/build/.cargo CARGO_INCREMENTAL=0 CARGO_MANIFEST_DIR=/work/oxidecomputer/omicron/nexus CARGO_PKG_AUTHORS= CARGO_PKG_DESCRIPTION= CARGO_PKG_HOMEPAGE= CARGO_PKG_LICENSE=MPL-2.0 CARGO_PKG_LICENSE_FILE= CARGO_PKG_NAME=omicron-nexus CARGO_PKG_REPOSITORY= CARGO_PKG_RUST_VERSION= CARGO_PKG_VERSION=0.1.0 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=1 CARGO_PKG_VERSION_PATCH=0 CARGO_PKG_VERSION_PRE= CARGO_TERM_COLOR=always CI=true CRDB_SEED_TAR=/var/tmp/omicron_tmp/crdb-base-build/715a51eb919177fa8d827f74e9ac217af9dd57c2e799d7d9e860efa5dba3a56f.tar GITHUB_BRANCH=qorb-terminate GITHUB_REF=refs/heads/qorb-terminate GITHUB_REPOSITORY=oxidecomputer/omicron GITHUB_SHA=95c68255ff52b8fc36544fe915226f9e7c07203b GOTRACEBACK=crash HOME=/home/build LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8 LD_LIBRARY_PATH=/work/oxidecomputer/omicron/target/debug/build/blake3-b5b7e9096c7345fc/out:/work/oxidecomputer/omicron/target/debug/build/bzip2-sys-9b85f27e0600ea92/out/lib:/work/oxidecomputer/omicron/target/debug/build/libgit2-sys-da9daf940bac384b/out/build:/work/oxidecomputer/omicron/target/debug/build/libz-sys-6acf30a23bb64bce/out/lib:/work/oxidecomputer/omicron/target/debug/build/ring-b091efe07f3181f9/out:/work/oxidecomputer/omicron/target/debug/build/ring-f37a71783cfb097f/out:/work/oxidecomputer/omicron/target/debug/build/tofino-3606fcc8e8df9f8f/out:/work/oxidecomputer/omicron/target/debug/deps:/work/oxidecomputer/omicron/target/debug:/home/build/.rustup/toolchains/1.80.1-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib:/home/build/.rustup/toolchains/1.80.1-x86_64-unknown-linux-gnu/lib LOGNAME=build NEXTEST=1 NEXTEST_BIN_EXE_nexus=/work/oxidecomputer/omicron/target/debug/nexus NEXTEST_BIN_EXE_schema-updater=/work/oxidecomputer/omicron/target/debug/schema-updater NEXTEST_EXECUTION_MODE=process-per-test NEXTEST_LD_LIBRARY_PATH=/work/oxidecomputer/omicron/target/debug/build/blake3-b5b7e9096c7345fc/out:/work/oxidecomputer/omicron/target/debug/build/bzip2-sys-9b85f27e0600ea92/out/lib:/work/oxidecomputer/omicron/target/debug/build/libgit2-sys-da9daf940bac384b/out/build:/work/oxidecomputer/omicron/target/debug/build/libz-sys-6acf30a23bb64bce/out/lib:/work/oxidecomputer/omicron/target/debug/build/ring-b091efe07f3181f9/out:/work/oxidecomputer/omicron/target/debug/build/ring-f37a71783cfb097f/out:/work/oxidecomputer/omicron/target/debug/build/tofino-3606fcc8e8df9f8f/out:/work/oxidecomputer/omicron/target/debug/deps:/work/oxidecomputer/omicron/target/debug:/home/build/.rustup/toolchains/1.80.1-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib:/home/build/.rustup/toolchains/1.80.1-x86_64-unknown-linux-gnu/lib NEXTEST_RUN_ID=5e435f7d-4fb4-4d4c-8776-80afd59afd06 OUT_DIR=/work/oxidecomputer/omicron/target/debug/build/omicron-nexus-d57ffda7b88b33e7/out PATH=/work/oxidecomputer/omicron/out/mgd/root/opt/oxide/mgd/bin:/work/oxidecomputer/omicron/out/dendrite-stub/bin:/work/oxidecomputer/omicron/out/clickhouse:/work/oxidecomputer/omicron/out/cockroachdb/bin:/home/build/.cargo/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/ooce/bin:/opt/ooce/sbin PWD=/work/oxidecomputer/omicron RUSTC_BOOTSTRAP=1 RUSTDOCFLAGS=--document-private-items -D warnings RUSTFLAGS=-D warnings RUSTUP_HOME=/home/build/.rustup RUSTUP_TOOLCHAIN=1.80.1-x86_64-unknown-linux-gnu RUST_BACKTRACE=1 RUST_RECURSION_COUNT=1 SHLVL=1 SSL_CERT_DIR=/usr/lib/ssl/certs SSL_CERT_FILE=/usr/lib/ssl/certs/ca-certificates.crt TMPDIR=/var/tmp/omicron_tmp TZ=UTC USER=build __NEXTEST_ATTEMPT=1 __NEXTEST_ORIGINAL_CARGO_MANIFEST_DIR=/work/oxidecomputer/omicron/nexus
72024-10-17T22:41:45.192ZINFOtest_metrics: cockroach pid: 149562
82024-10-17T22:41:45.192ZINFOtest_metrics: cockroach listen URL: postgresql://root@[::1]:41953/omicron?sslmode=disable
92024-10-17T22:41:45.192ZDEBGtest_metrics (init_with_steps): Running step start_clickhouse
102024-10-17T22:41:45.192ZDEBGtest_metrics: Starting Clickhouse
112024-10-17T22:41:45.835ZDEBGtest_metrics (init_with_steps): Running step start_gateway_switch0
122024-10-17T22:41:45.835ZDEBGtest_metrics: Starting Management Gateway
132024-10-17T22:41:45.970ZDEBGtest_metrics (init_with_steps): Running step start_gateway_switch1
142024-10-17T22:41:45.971ZDEBGtest_metrics: Starting Management Gateway
152024-10-17T22:41:46.088ZDEBGtest_metrics (init_with_steps): Running step start_dendrite_switch0
162024-10-17T22:41:46.088ZDEBGtest_metrics: Starting Dendrite for switch0
172024-10-17T22:41:46.120ZDEBGtest_metrics (init_with_steps): Running step start_dendrite_switch1
182024-10-17T22:41:46.120ZDEBGtest_metrics: Starting Dendrite for switch1
192024-10-17T22:41:46.148ZDEBGtest_metrics (init_with_steps): Running step start_mgd_switch0
202024-10-17T22:41:46.148ZDEBGtest_metrics: Starting mgd for switch0
212024-10-17T22:41:46.173ZDEBGtest_metrics: mgd port is 39577
222024-10-17T22:41:46.174ZDEBGtest_metrics (init_with_steps): Running step start_mgd_switch1
232024-10-17T22:41:46.174ZDEBGtest_metrics: Starting mgd for switch1
242024-10-17T22:41:46.204ZDEBGtest_metrics: mgd port is 37593
252024-10-17T22:41:46.204ZDEBGtest_metrics (init_with_steps): Running step record_switch_dns
262024-10-17T22:41:46.204ZDEBGtest_metrics: Recording DNS for the switch zones
272024-10-17T22:41:46.204ZDEBGtest_metrics (init_with_steps): Running step start_internal_dns
282024-10-17T22:41:46.204ZINFOtest_metrics (store): opening sled database path = /var/tmp/omicron_tmp/.tmp4JAOBq
292024-10-17T22:41:46.263ZINFOtest_metrics (store): pruning trees for generations newer than 0
302024-10-17T22:41:46.264ZINFOtest_metrics (store): pruning trees for generations older than 0 keep = 3
312024-10-17T22:41:46.264ZINFOtest_metrics (dns): DNS server bound to address kind = dns local_address = [::1]:41439
322024-10-17T22:41:46.264ZDEBGtest_metrics (http): registered endpoint kind = dns local_addr = [::1]:41599 method = GET path = /config
332024-10-17T22:41:46.264ZDEBGtest_metrics (http): registered endpoint kind = dns local_addr = [::1]:41599 method = PUT path = /config
342024-10-17T22:41:46.264ZINFOtest_metrics (http): listening kind = dns local_addr = [::1]:41599
352024-10-17T22:41:46.264ZDEBGtest_metrics (http): successfully registered DTrace USDT probes kind = dns local_addr = [::1]:41599
362024-10-17T22:41:46.264ZDEBGtest_metrics (init_with_steps): Running step start_external_dns
372024-10-17T22:41:46.264ZINFOtest_metrics (store): opening sled database path = /var/tmp/omicron_tmp/.tmpeYZ6Xk
382024-10-17T22:41:46.265ZINFOtest_metrics (store): pruning trees for generations newer than 0
392024-10-17T22:41:46.265ZINFOtest_metrics (store): pruning trees for generations older than 0 keep = 3
402024-10-17T22:41:46.265ZINFOtest_metrics (dns): DNS server bound to address kind = dns local_address = [::1]:37462
412024-10-17T22:41:46.265ZDEBGtest_metrics (http): registered endpoint kind = dns local_addr = [::1]:36177 method = GET path = /config
422024-10-17T22:41:46.265ZDEBGtest_metrics (http): registered endpoint kind = dns local_addr = [::1]:36177 method = PUT path = /config
432024-10-17T22:41:46.265ZINFOtest_metrics (http): listening kind = dns local_addr = [::1]:36177
442024-10-17T22:41:46.265ZDEBGtest_metrics (http): successfully registered DTrace USDT probes kind = dns local_addr = [::1]:36177
452024-10-17T22:41:46.265ZDEBGtest_metrics (init_with_steps): Running step start_nexus_internal
462024-10-17T22:41:46.265ZDEBGtest_metrics: Starting Nexus (internal API)
472024-10-17T22:41:46.265ZINFOtest_metrics: setting up nexus server
482024-10-17T22:41:46.266ZINFOtest_metrics (ServerContext): registering Oso class class = Action
492024-10-17T22:41:46.266ZINFOtest_metrics (ServerContext): registering Oso class class = AnyActor
502024-10-17T22:41:46.266ZINFOtest_metrics (ServerContext): registering Oso class class = AuthenticatedActor
512024-10-17T22:41:46.266ZINFOtest_metrics (ServerContext): registering Oso class class = BlueprintConfig
522024-10-17T22:41:46.266ZINFOtest_metrics (ServerContext): registering Oso class class = Database
532024-10-17T22:41:46.266ZINFOtest_metrics (ServerContext): registering Oso class class = DnsConfig
542024-10-17T22:41:46.266ZINFOtest_metrics (ServerContext): registering Oso class class = Fleet
552024-10-17T22:41:46.266ZINFOtest_metrics (ServerContext): registering Oso class class = Inventory
562024-10-17T22:41:46.266ZINFOtest_metrics (ServerContext): registering Oso class class = IpPoolList
572024-10-17T22:41:46.266ZINFOtest_metrics (ServerContext): registering Oso class class = ConsoleSessionList
582024-10-17T22:41:46.266ZINFOtest_metrics (ServerContext): registering Oso class class = DeviceAuthRequestList
592024-10-17T22:41:46.266ZINFOtest_metrics (ServerContext): registering Oso class class = SiloCertificateList
602024-10-17T22:41:46.266ZINFOtest_metrics (ServerContext): registering Oso class class = SiloIdentityProviderList
612024-10-17T22:41:46.266ZINFOtest_metrics (ServerContext): registering Oso class class = SiloUserList
622024-10-17T22:41:46.266ZINFOtest_metrics (ServerContext): registering Oso class class = Project
632024-10-17T22:41:46.266ZINFOtest_metrics (ServerContext): registering Oso class class = Disk
642024-10-17T22:41:46.266ZINFOtest_metrics (ServerContext): registering Oso class class = Snapshot
652024-10-17T22:41:46.266ZINFOtest_metrics (ServerContext): registering Oso class class = ProjectImage
662024-10-17T22:41:46.266ZINFOtest_metrics (ServerContext): registering Oso class class = Instance
672024-10-17T22:41:46.266ZINFOtest_metrics (ServerContext): registering Oso class class = IpPool
682024-10-17T22:41:46.266ZINFOtest_metrics (ServerContext): registering Oso class class = InstanceNetworkInterface
692024-10-17T22:41:46.266ZINFOtest_metrics (ServerContext): registering Oso class class = Vpc
702024-10-17T22:41:46.267ZINFOtest_metrics (ServerContext): registering Oso class class = VpcRouter
712024-10-17T22:41:46.267ZINFOtest_metrics (ServerContext): registering Oso class class = InternetGateway
722024-10-17T22:41:46.267ZINFOtest_metrics (ServerContext): registering Oso class class = InternetGatewayIpPool
732024-10-17T22:41:46.267ZINFOtest_metrics (ServerContext): registering Oso class class = InternetGatewayIpAddress
742024-10-17T22:41:46.267ZINFOtest_metrics (ServerContext): registering Oso class class = RouterRoute
752024-10-17T22:41:46.267ZINFOtest_metrics (ServerContext): registering Oso class class = VpcSubnet
762024-10-17T22:41:46.267ZINFOtest_metrics (ServerContext): registering Oso class class = FloatingIp
772024-10-17T22:41:46.267ZINFOtest_metrics (ServerContext): registering Oso class class = Image
782024-10-17T22:41:46.267ZINFOtest_metrics (ServerContext): registering Oso class class = SiloImage
792024-10-17T22:41:46.267ZINFOtest_metrics (ServerContext): registering Oso class class = AddressLot
802024-10-17T22:41:46.267ZINFOtest_metrics (ServerContext): registering Oso class class = Blueprint
812024-10-17T22:41:46.267ZINFOtest_metrics (ServerContext): registering Oso class class = LoopbackAddress
822024-10-17T22:41:46.267ZINFOtest_metrics (ServerContext): registering Oso class class = Certificate
832024-10-17T22:41:46.267ZINFOtest_metrics (ServerContext): registering Oso class class = ConsoleSession
842024-10-17T22:41:46.267ZINFOtest_metrics (ServerContext): registering Oso class class = DeviceAuthRequest
852024-10-17T22:41:46.267ZINFOtest_metrics (ServerContext): registering Oso class class = DeviceAccessToken
862024-10-17T22:41:46.267ZINFOtest_metrics (ServerContext): registering Oso class class = PhysicalDisk
872024-10-17T22:41:46.267ZINFOtest_metrics (ServerContext): registering Oso class class = Rack
882024-10-17T22:41:46.268ZINFOtest_metrics (ServerContext): registering Oso class class = RoleBuiltin
892024-10-17T22:41:46.268ZINFOtest_metrics (ServerContext): registering Oso class class = SshKey
902024-10-17T22:41:46.268ZINFOtest_metrics (ServerContext): registering Oso class class = Silo
912024-10-17T22:41:46.268ZINFOtest_metrics (ServerContext): registering Oso class class = SiloUser
922024-10-17T22:41:46.268ZINFOtest_metrics (ServerContext): registering Oso class class = SiloGroup
932024-10-17T22:41:46.268ZINFOtest_metrics (ServerContext): registering Oso class class = IdentityProvider
942024-10-17T22:41:46.268ZINFOtest_metrics (ServerContext): registering Oso class class = SamlIdentityProvider
952024-10-17T22:41:46.268ZINFOtest_metrics (ServerContext): registering Oso class class = Sled
962024-10-17T22:41:46.268ZINFOtest_metrics (ServerContext): registering Oso class class = TufRepo
972024-10-17T22:41:46.268ZINFOtest_metrics (ServerContext): registering Oso class class = TufArtifact
982024-10-17T22:41:46.268ZINFOtest_metrics (ServerContext): registering Oso class class = Zpool
992024-10-17T22:41:46.268ZINFOtest_metrics (ServerContext): registering Oso class class = Service
1002024-10-17T22:41:46.268ZINFOtest_metrics (ServerContext): registering Oso class class = UserBuiltin
1012024-10-17T22:41:46.269ZINFOtest_metrics (ServerContext): full Oso configuration config = #\n# Oso configuration for Omicron\n# This file is augmented by generated snippets.\n#\n\n#\n# ACTOR TYPES AND BASIC RULES\n#\n\n# `AnyActor` includes both authenticated and unauthenticated users.\nactor AnyActor {}\n\n# An `AuthenticatedActor` has an identity in the system. All of our operations\n# today require that an actor be authenticated.\nactor AuthenticatedActor {}\n\n# For any resource, `actor` can perform action `action` on it if they're\n# authenticated and their role(s) give them the corresponding permission on that\n# resource.\nallow(actor: AnyActor, action: Action, resource) if\n actor.authenticated and\n has_permission(actor.authn_actor.unwrap(), action.to_perm(), resource);\n\n# Define role relationships\nhas_role(actor: AuthenticatedActor, role: String, resource: Resource)\n\tif resource.has_role(actor, role);\n\n#\n# ROLES AND PERMISSIONS IN THE FLEET/SILO/PROJECT HIERARCHY\n#\n# We define the following permissions for most resources in the system:\n#\n# - "create_child": required to create child resources (of any type)\n#\n# - "list_children": required to list child resources (of all types) of a\n# resource\n#\n# - "modify": required to modify or delete a resource\n#\n# - "read": required to read a resource\n#\n# We define the following predefined roles for only a few high-level resources:\n# the Fleet (see below), Silo, Organization, and Project. The specific roles\n# are oriented around intended use-cases:\n#\n# - "admin": has all permissions on the resource\n#\n# - "collaborator": has "read", "list_children", and "create_child", plus\n# the "admin" role for child resources. The idea is that if you're an\n# Organization Collaborator, you have full control over the Projects within\n# the Organization, but you cannot modify or delete the Organization itself.\n#\n# - "viewer": has "read" and "list_children" on a resource\n#\n# Below the Project level, permissions are granted via roles at the Project\n# level. For example, for someone to be able to create, modify, or delete any\n# Instances, they must be granted project.collaborator, which means they can\n# create, modify, or delete _all_ resources in the Project.\n#\n# The complete set of predefined roles:\n#\n# - fleet.admin (superuser for the whole system)\n# - fleet.collaborator (can manage Silos)\n# - fleet.viewer (can read most non-siloed resources in the system)\n# - silo.admin (superuser for the silo)\n# - silo.collaborator (can create and own Organizations)\n# - silo.viewer (can read most resources within the Silo)\n# - organization.admin (complete control over an organization)\n# - organization.collaborator (can manage Projects)\n# - organization.viewer (can read most resources within the Organization)\n# - project.admin (complete control over a Project)\n# - project.collaborator (can manage all resources within the Project)\n# - project.viewer (can read most resources within the Project)\n#\n# Outside the Silo/Organization/Project hierarchy, we (currently) treat most\n# resources as nested under Fleet or else a synthetic resource (see below). We\n# do not yet support role assignments on anything other than Fleet, Silo,\n# Organization, or Project.\n#\n\n# "Fleet" is a global singleton representing the whole system. The name comes\n# from the idea described in RFD 24, but it's not quite right. This probably\n# should be more like "Region" or "AvailabilityZone". The precise boundaries\n# have not yet been figured out.\nresource Fleet {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\troles = [\n\t # Roles that can be attached by users\n\t "admin",\n\t "collaborator",\n\t "viewer",\n\n\t # Internal-only roles\n\t "external-authenticator"\n\t];\n\n\t# Roles implied by other roles on this resource\n\t"viewer" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\t"create_child" if "collaborator";\n\t"modify" if "admin";\n}\n\n# For fleets specifically, roles can be conferred by roles on the user's Silo.\nhas_role(actor: AuthenticatedActor, role: String, _: Fleet) if\n\tsilo_role in actor.confers_fleet_role(role) and\n\thas_role(actor, silo_role, actor.silo.unwrap());\n\nresource Silo {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\troles = [ "admin", "collaborator", "viewer" ];\n\n\t# Roles implied by other roles on this resource\n\t"viewer" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\n\t"create_child" if "collaborator";\n\t"modify" if "admin";\n\n\t# Permissions implied by roles on this resource's parent (Fleet). Fleet\n\t# privileges allow a user to see and potentially administer the Silo,\n\t# but they do not give anyone permission to look at anything inside the\n\t# Silo. To achieve this, we use permission rules here. (If we granted\n\t# Fleet administrators _roles_ on the Silo, then those would cascade\n\t# into the Silo as well.)\n\trelations = { parent_fleet: Fleet };\n\t"read" if "viewer" on "parent_fleet";\n\t"modify" if "collaborator" on "parent_fleet";\n\n\t# external authenticator has to create silo users\n\t"list_children" if "external-authenticator" on "parent_fleet";\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\n\nhas_relation(fleet: Fleet, "parent_fleet", silo: Silo)\n\tif silo.fleet = fleet;\n\n# As a special case, all authenticated users can read their own Silo. That's\n# not quite the same as having the "viewer" role. For example, they cannot list\n# Organizations in the Silo.\n#\n# One reason this is necessary is because if an unprivileged user tries to\n# create an Organization using "POST /organizations", they should get back a 403\n# (which implies they're able to see /organizations, which is essentially seeing\n# the Silo itself) rather than a 404. This behavior isn't a hard constraint\n# (i.e., you could reasonably get a 404 for an API you're not allowed to call).\n# Nor is the implementation (i.e., we could special-case this endpoint somehow).\n# But granting this permission is the simplest way to keep this endpoint's\n# behavior consistent with the rest of the API.\n#\n# This rule is also used to determine if a user can list the identity providers\n# in the Silo (which they should be able to), since that's predicated on being\n# able to read the Silo.\n#\n# It's unclear what else would break if users couldn't see their own Silo.\nhas_permission(actor: AuthenticatedActor, "read", silo: Silo)\n\tif silo in actor.silo;\n\nresource Project {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\troles = [ "admin", "collaborator", "viewer" ];\n\n\t# Roles implied by other roles on this resource\n\t"viewer" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\t"create_child" if "collaborator";\n\t"modify" if "admin";\n\n\t# Roles implied by roles on this resource's parent (Silo)\n\trelations = { parent_silo: Silo };\n\t"admin" if "collaborator" on "parent_silo";\n\t"viewer" if "viewer" on "parent_silo";\n}\nhas_relation(silo: Silo, "parent_silo", project: Project)\n\tif project.silo = silo;\n\n#\n# GENERAL RESOURCES OUTSIDE THE SILO/PROJECT HIERARCHY\n#\n# Many resources use snippets of Polar generated by the `authz_resource!` Rust\n# macro. Some resources require custom Polar code. Those appear here.\n#\n\nresource Certificate {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Fleet-level and silo-level roles both grant privileges on certificates.\n\t"read" if "admin" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"read" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", certificate: Certificate)\n\tif certificate.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", certificate: Certificate)\n\tif certificate.silo.fleet = fleet;\n\nresource SiloUser {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\t# Fleet and Silo administrators can manage a Silo's users. This is one\n\t# of the only areas of Silo configuration that Fleet Administrators have\n\t# permissions on.\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\t"list_children" if "read" on "parent_silo";\n\t"read" if "read" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\t"list_children" if "read" on "parent_fleet";\n\t"read" if "read" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", user: SiloUser)\n\tif user.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", user: SiloUser)\n\tif user.silo.fleet = fleet;\n\n# authenticated actors have all permissions on themselves\nhas_permission(actor: AuthenticatedActor, _perm: String, silo_user: SiloUser)\n if actor.equals_silo_user(silo_user);\n\nhas_permission(actor: AuthenticatedActor, "read", silo_user: SiloUser)\n if silo_user.silo in actor.silo;\n\nresource SiloGroup {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\trelations = { parent_silo: Silo };\n\t"list_children" if "read" on "parent_silo";\n\t"read" if "read" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n}\nhas_relation(silo: Silo, "parent_silo", group: SiloGroup)\n\tif group.silo = silo;\n\nresource SshKey {\n\tpermissions = [ "read", "modify" ];\n\trelations = { silo_user: SiloUser };\n\n\t"read" if "read" on "silo_user";\n\t"modify" if "modify" on "silo_user";\n}\nhas_relation(user: SiloUser, "silo_user", ssh_key: SshKey)\n\tif ssh_key.silo_user = user;\n\nresource IdentityProvider {\n\tpermissions = [\n\t "read",\n\t "modify",\n\t "create_child",\n\t "list_children",\n\t];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Silo-level roles grant privileges on identity providers.\n\t"read" if "viewer" on "parent_silo";\n\t"list_children" if "viewer" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\n\t# Fleet-level roles also grant privileges on identity providers.\n\t"read" if "viewer" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", identity_provider: IdentityProvider)\n\tif identity_provider.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: IdentityProvider)\n\tif collection.silo.fleet = fleet;\n\nresource SamlIdentityProvider {\n\tpermissions = [\n\t "read",\n\t "modify",\n\t "create_child",\n\t "list_children",\n\t];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Silo-level roles grant privileges on identity providers.\n\t"read" if "viewer" on "parent_silo";\n\t"list_children" if "viewer" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\n\t# Fleet-level roles also grant privileges on identity providers.\n\t"read" if "viewer" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", saml_identity_provider: SamlIdentityProvider)\n\tif saml_identity_provider.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SamlIdentityProvider)\n\tif collection.silo.fleet = fleet;\n\n#\n# SYNTHETIC RESOURCES OUTSIDE THE SILO HIERARCHY\n#\n# The resources here do not correspond to anything that appears explicitly in\n# the API or is stored in the database. These are used either at the top level\n# of the API path (e.g., "/v1/system/ip-pools") or as an implementation detail of the system\n# (in the case of console sessions and "Database"). The policies are\n# either statically-defined in this file or driven by role assignments on the\n# Fleet. None of these resources defines their own roles.\n#\n\n# Describes the policy for reading and modifying DNS configuration\n# (both internal and external)\nresource DnsConfig {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_fleet: Fleet };\n\t# "external-authenticator" requires these permissions because that's the\n\t# context that Nexus uses when creating and deleting Silos. These\n\t# operations necessarily need to read and modify DNS configuration.\n\t"read" if "external-authenticator" on "parent_fleet";\n\t"modify" if "external-authenticator" on "parent_fleet";\n\t# "admin" on the parent fleet also gets these permissions, primarily for\n\t# the test suite.\n\t"read" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", dns_config: DnsConfig)\n\tif dns_config.fleet = fleet;\n\n# Describes the policy for accessing blueprints\nresource BlueprintConfig {\n\tpermissions = [\n\t "list_children", # list blueprints\n\t "create_child", # create blueprint\n\t "read", # read the current target\n\t "modify", # change the current target\n\t];\n\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"read" if "viewer" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", list: BlueprintConfig)\n\tif list.fleet = fleet;\n\n# Describes the policy for reading and modifying low-level inventory\nresource Inventory {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_fleet: Fleet };\n\t"read" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", inventory: Inventory)\n\tif inventory.fleet = fleet;\n\n# Describes the policy for accessing "/v1/system/ip-pools" in the API\nresource IpPoolList {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "create_child",\n\t];\n\n\t# Fleet Administrators can create or modify the IP Pools list.\n\trelations = { parent_fleet: Fleet };\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n\n\t# Fleet Viewers can list IP Pools\n\t"list_children" if "viewer" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", ip_pool_list: IpPoolList)\n\tif ip_pool_list.fleet = fleet;\n\n# Any authenticated user can create a child of a provided IP Pool.\n# This is necessary to use the pools when provisioning instances.\nhas_permission(actor: AuthenticatedActor, "create_child", ip_pool: IpPool)\n\tif silo in actor.silo and silo.fleet = ip_pool.fleet;\n\n# Describes the policy for creating and managing web console sessions.\nresource ConsoleSessionList {\n\tpermissions = [ "create_child" ];\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", collection: ConsoleSessionList)\n\tif collection.fleet = fleet;\n\n# Describes the policy for creating and managing device authorization requests.\nresource DeviceAuthRequestList {\n\tpermissions = [ "create_child" ];\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", collection: DeviceAuthRequestList)\n\tif collection.fleet = fleet;\n\n# Describes the policy for creating and managing Silo certificates\nresource SiloCertificateList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Both Fleet and Silo administrators can see and modify the Silo's\n\t# certificates.\n\t"list_children" if "admin" on "parent_silo";\n\t"list_children" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloCertificateList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloCertificateList)\n\tif collection.silo.fleet = fleet;\n\n# Describes the policy for creating and managing Silo identity providers\nresource SiloIdentityProviderList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Everyone who can read the Silo (which includes all the users in the\n\t# Silo) can see the identity providers in it.\n\t"list_children" if "read" on "parent_silo";\n\n\t# Fleet and Silo administrators can manage the Silo's identity provider\n\t# configuration. This is one of the only areas of Silo configuration\n\t# that Fleet Administrators have permissions on. This is also one of\n\t# the only cases where we need to look two levels up the hierarchy to\n\t# see if somebody has the right permission. For most other things,\n\t# permissions cascade down the hierarchy so we only need to look at the\n\t# parent.\n\t"create_child" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloIdentityProviderList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloIdentityProviderList)\n\tif collection.silo.fleet = fleet;\n\n# Describes the policy for creating and managing Silo users (mostly intended for\n# API-managed users)\nresource SiloUserList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Everyone who can read the Silo (which includes all the users in the\n\t# Silo) can see the users in it.\n\t"list_children" if "read" on "parent_silo";\n\n\t# Fleet and Silo administrators can manage the Silo's users. This is\n\t# one of the only areas of Silo configuration that Fleet Administrators\n\t# have permissions on. This is also one of the few cases (so far) where\n\t# we need to look two levels up the hierarchy to see if somebody has the\n\t# right permission. For most other things, permissions cascade down the\n\t# hierarchy so we only need to look at the parent.\n\t"create_child" if "admin" on "parent_silo";\n\t"list_children" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloUserList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloUserList)\n\tif collection.silo.fleet = fleet;\n\n# These rules grants the external authenticator role the permissions it needs to\n# read silo users and modify their sessions. This is necessary for login to\n# work.\nhas_permission(actor: AuthenticatedActor, "read", silo: Silo)\n\tif has_role(actor, "external-authenticator", silo.fleet);\nhas_permission(actor: AuthenticatedActor, "read", user: SiloUser)\n\tif has_role(actor, "external-authenticator", user.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", user: SiloUser)\n\tif has_role(actor, "external-authenticator", user.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "read", group: SiloGroup)\n\tif has_role(actor, "external-authenticator", group.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", group: SiloGroup)\n\tif has_role(actor, "external-authenticator", group.silo.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", session: ConsoleSession)\n\tif has_role(actor, "external-authenticator", session.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", session: ConsoleSession)\n\tif has_role(actor, "external-authenticator", session.fleet);\n\n# All authenticated users can read and delete device authn requests because\n# by necessity these operations happen before we've figured out what user (or\n# even Silo) the device auth is associated with. Any user can claim a device\n# auth request with the right user code (that's how it works) -- it's the user\n# code and associated logic that prevents unauthorized access here.\nhas_permission(_actor: AuthenticatedActor, "read", _device_auth: DeviceAuthRequest);\nhas_permission(_actor: AuthenticatedActor, "modify", _device_auth: DeviceAuthRequest);\n\nhas_permission(actor: AuthenticatedActor, "read", device_token: DeviceAccessToken)\n\tif has_role(actor, "external-authenticator", device_token.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", identity_provider: IdentityProvider)\n\tif has_role(actor, "external-authenticator", identity_provider.silo.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", saml_identity_provider: SamlIdentityProvider)\n\tif has_role(actor, "external-authenticator", saml_identity_provider.silo.fleet);\n\n# Describes the policy for who can access the internal database.\nresource Database {\n\tpermissions = [\n\t # "query" is required to perform any query against the database,\n\t # whether a read or write query. This is checked when an operation\n\t # checks out a database connection from the connection pool.\n\t #\n\t # Any authenticated user gets this permission. There's generally\n\t # some other authz check involved in the database query. For\n\t # example, if you're querying the database to "read" a "Project", we\n\t # should also be checking that. So why do we do this at all? It's\n\t # a belt-and-suspenders measure so that if we somehow introduced an\n\t # unauthenticated code path that hits the database, it cannot be\n\t # used to DoS the database because we won't allow the operation to\n\t # make the query. (As long as the code path _is_ authenticated, we\n\t # can use throttling mechanisms to prevent DoS.)\n\t "query",\n\n\t # "modify" is required to populate database data that's delivered\n\t # with the system. It should also be required for schema changes,\n\t # when we support those. This is separate from "query" so that we\n\t # cannot accidentally invoke these code paths from API calls and\n\t # other general functions.\n\t "modify"\n\t];\n}\n\n# All authenticated users have the "query" permission on the database.\nhas_permission(_actor: AuthenticatedActor, "query", _resource: Database);\n\n# The "db-init" user is the only one with the "modify" permission.\nhas_permission(USER_DB_INIT: AuthenticatedActor, "modify", _resource: Database);\nhas_permission(USER_DB_INIT: AuthenticatedActor, "create_child", _resource: IpPoolList);\n# It also has "admin" on the internal silo to populate it with built-in resources.\n# TODO-completeness: actually limit to just internal silo and not all silos\nhas_role(USER_DB_INIT: AuthenticatedActor, "admin", _silo: Silo);\n\n# Allow the internal API admin permissions on all silos.\nhas_role(USER_INTERNAL_API: AuthenticatedActor, "admin", _silo: Silo);\n\n\n\n resource Disk {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Disk)\n if child.project = parent;\n \n\n resource Snapshot {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Snapshot)\n if child.project = parent;\n \n\n resource ProjectImage {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: ProjectImage)\n if child.project = parent;\n \n\n resource Instance {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Instance)\n if child.project = parent;\n \n\n resource IpPool {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: IpPool)\n if child.fleet = fleet;\n \n\n resource InstanceNetworkInterface {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Instance\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InstanceNetworkInterface)\n if has_relation(project, "containing_project", child.instance);\n\n has_relation(parent: Instance, "parent", child: InstanceNetworkInterface)\n if child.instance = parent;\n \n\n resource Vpc {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Vpc)\n if child.project = parent;\n \n\n resource VpcRouter {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Vpc\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: VpcRouter)\n if has_relation(project, "containing_project", child.vpc);\n\n has_relation(parent: Vpc, "parent", child: VpcRouter)\n if child.vpc = parent;\n \n\n resource InternetGateway {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Vpc\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InternetGateway)\n if has_relation(project, "containing_project", child.vpc);\n\n has_relation(parent: Vpc, "parent", child: InternetGateway)\n if child.vpc = parent;\n \n\n resource InternetGatewayIpPool {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: InternetGateway\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InternetGatewayIpPool)\n if has_relation(project, "containing_project", child.internet_gateway);\n\n has_relation(parent: InternetGateway, "parent", child: InternetGatewayIpPool)\n if child.internet_gateway = parent;\n \n\n resource InternetGatewayIpAddress {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: InternetGateway\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InternetGatewayIpAddress)\n if has_relation(project, "containing_project", child.internet_gateway);\n\n has_relation(parent: InternetGateway, "parent", child: InternetGatewayIpAddress)\n if child.internet_gateway = parent;\n \n\n resource RouterRoute {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: VpcRouter\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: RouterRoute)\n if has_relation(project, "containing_project", child.vpc_router);\n\n has_relation(parent: VpcRouter, "parent", child: RouterRoute)\n if child.vpc_router = parent;\n \n\n resource VpcSubnet {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Vpc\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: VpcSubnet)\n if has_relation(project, "containing_project", child.vpc);\n\n has_relation(parent: Vpc, "parent", child: VpcSubnet)\n if child.vpc = parent;\n \n\n resource FloatingIp {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: FloatingIp)\n if child.project = parent;\n \n\n resource Image {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_silo: Silo };\n "list_children" if "viewer" on "containing_silo";\n "read" if "viewer" on "containing_silo";\n "modify" if "collaborator" on "containing_silo";\n "create_child" if "collaborator" on "containing_silo";\n }\n\n has_relation(parent: Silo, "containing_silo", child: Image)\n if child.silo = parent;\n \n\n resource SiloImage {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_silo: Silo };\n "list_children" if "viewer" on "containing_silo";\n "read" if "viewer" on "containing_silo";\n "modify" if "collaborator" on "containing_silo";\n "create_child" if "collaborator" on "containing_silo";\n }\n\n has_relation(parent: Silo, "containing_silo", child: SiloImage)\n if child.silo = parent;\n \n\n resource AddressLot {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: AddressLot)\n if child.fleet = fleet;\n \n\n resource Blueprint {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Blueprint)\n if child.fleet = fleet;\n \n\n resource LoopbackAddress {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: LoopbackAddress)\n if child.fleet = fleet;\n \n\n\n resource ConsoleSession {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: ConsoleSession)\n if child.fleet = fleet;\n \n\n resource DeviceAuthRequest {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: DeviceAuthRequest)\n if child.fleet = fleet;\n \n\n resource DeviceAccessToken {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: DeviceAccessToken)\n if child.fleet = fleet;\n \n\n resource PhysicalDisk {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: PhysicalDisk)\n if child.fleet = fleet;\n \n\n resource Rack {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Rack)\n if child.fleet = fleet;\n \n\n resource RoleBuiltin {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: RoleBuiltin)\n if child.fleet = fleet;\n \n\n\n\n\n\n\n\n resource Sled {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Sled)\n if child.fleet = fleet;\n \n\n resource TufRepo {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: TufRepo)\n if child.fleet = fleet;\n \n\n resource TufArtifact {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: TufArtifact)\n if child.fleet = fleet;\n \n\n resource Zpool {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Zpool)\n if child.fleet = fleet;\n \n\n resource Service {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Service)\n if child.fleet = fleet;\n \n\n resource UserBuiltin {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: UserBuiltin)\n if child.fleet = fleet;\n
1022024-10-17T22:41:46.283ZINFOtest_metrics (ServerContext): Setting up resolver using DNS address: [::1]:41439
1032024-10-17T22:41:46.284ZINFOtest_metrics (DnsResolver): new DNS resolver addresses = [[::1]:41439]
1042024-10-17T22:41:46.284ZINFOtest_metrics (ServerContext): Setting up qorb database pool from a single host url = PostgresConfigWithUrl {\n url_raw: "postgresql://root@[::1]:41953/omicron?sslmode=disable",\n config: Config {\n user: Some(\n "root",\n ),\n password: None,\n dbname: Some(\n "omicron",\n ),\n options: None,\n application_name: None,\n ssl_mode: Disable,\n host: [\n Tcp(\n "::1",\n ),\n ],\n hostaddr: [],\n port: [\n 41953,\n ],\n connect_timeout: None,\n tcp_user_timeout: None,\n keepalives: true,\n keepalives_idle: 7200s,\n keepalives_interval: None,\n keepalives_retries: None,\n target_session_attrs: Any,\n channel_binding: Prefer,\n },\n}
1052024-10-17T22:41:46.313ZINFOtest_metrics (nexus): Database schema version is up to date desired_version = 109.0.0 found_version = 109.0.0
1062024-10-17T22:41:46.457ZDEBGtest_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:33311 method = GET path = /artifacts/{kind}/{name}/{version}
1072024-10-17T22:41:46.457ZDEBGtest_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:33311 method = GET path = /bgtasks
1082024-10-17T22:41:46.457ZDEBGtest_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:33311 method = POST path = /bgtasks/activate
1092024-10-17T22:41:46.457ZDEBGtest_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:33311 method = GET path = /bgtasks/view/{bgtask_name}
1102024-10-17T22:41:46.457ZDEBGtest_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:33311 method = POST path = /crucible/0/upstairs/{upstairs_id}/downstairs/{downstairs_id}/stop-request
1112024-10-17T22:41:46.457ZDEBGtest_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:33311 method = POST path = /crucible/0/upstairs/{upstairs_id}/downstairs/{downstairs_id}/stopped
1122024-10-17T22:41:46.457ZDEBGtest_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:33311 method = POST path = /crucible/0/upstairs/{upstairs_id}/repair/{repair_id}/progress
1132024-10-17T22:41:46.457ZDEBGtest_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:33311 method = POST path = /crucible/0/upstairs/{upstairs_id}/repair-finish
1142024-10-17T22:41:46.457ZDEBGtest_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:33311 method = POST path = /crucible/0/upstairs/{upstairs_id}/repair-start
1152024-10-17T22:41:46.457ZDEBGtest_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:33311 method = POST path = /demo-saga
1162024-10-17T22:41:46.457ZDEBGtest_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:33311 method = POST path = /demo-saga/{demo_saga_id}/complete
1172024-10-17T22:41:46.457ZDEBGtest_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:33311 method = GET path = /deployment/blueprints/all
1182024-10-17T22:41:46.457ZDEBGtest_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:33311 method = DELETE path = /deployment/blueprints/all/{blueprint_id}
1192024-10-17T22:41:46.457ZDEBGtest_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:33311 method = GET path = /deployment/blueprints/all/{blueprint_id}
1202024-10-17T22:41:46.457ZDEBGtest_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:33311 method = POST path = /deployment/blueprints/import
1212024-10-17T22:41:46.457ZDEBGtest_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:33311 method = POST path = /deployment/blueprints/regenerate
1222024-10-17T22:41:46.457ZDEBGtest_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:33311 method = GET path = /deployment/blueprints/target
1232024-10-17T22:41:46.457ZDEBGtest_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:33311 method = POST path = /deployment/blueprints/target
1242024-10-17T22:41:46.457ZDEBGtest_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:33311 method = PUT path = /deployment/blueprints/target/enabled
1252024-10-17T22:41:46.458ZDEBGtest_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:33311 method = POST path = /disk/{disk_id}/remove-read-only-parent
1262024-10-17T22:41:46.458ZDEBGtest_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:33311 method = PUT path = /disks/{disk_id}
1272024-10-17T22:41:46.458ZDEBGtest_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:33311 method = POST path = /instances/{instance_id}/migrate
1282024-10-17T22:41:46.458ZDEBGtest_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:33311 method = POST path = /metrics/collectors
1292024-10-17T22:41:46.458ZDEBGtest_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:33311 method = GET path = /metrics/collectors/{collector_id}/producers
1302024-10-17T22:41:46.458ZDEBGtest_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:33311 method = POST path = /metrics/producers
1312024-10-17T22:41:46.458ZDEBGtest_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:33311 method = GET path = /nat/ipv4/changeset/{from_gen}
1322024-10-17T22:41:46.458ZDEBGtest_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:33311 method = POST path = /physical-disk/expunge
1332024-10-17T22:41:46.458ZDEBGtest_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:33311 method = GET path = /probes/{sled}
1342024-10-17T22:41:46.458ZDEBGtest_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:33311 method = PUT path = /racks/{rack_id}/initialization-complete
1352024-10-17T22:41:46.458ZDEBGtest_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:33311 method = GET path = /sagas
1362024-10-17T22:41:46.458ZDEBGtest_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:33311 method = GET path = /sagas/{saga_id}
1372024-10-17T22:41:46.458ZDEBGtest_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:33311 method = GET path = /sled-agents/{sled_id}
1382024-10-17T22:41:46.458ZDEBGtest_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:33311 method = POST path = /sled-agents/{sled_id}
1392024-10-17T22:41:46.458ZDEBGtest_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:33311 method = POST path = /sled-agents/{sled_id}/firewall-rules-update
1402024-10-17T22:41:46.458ZDEBGtest_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:33311 method = POST path = /sleds/add
1412024-10-17T22:41:46.458ZDEBGtest_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:33311 method = POST path = /sleds/expunge
1422024-10-17T22:41:46.459ZDEBGtest_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:33311 method = GET path = /sleds/uninitialized
1432024-10-17T22:41:46.459ZDEBGtest_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:33311 method = PUT path = /switch/{switch_id}
1442024-10-17T22:41:46.460ZDEBGtest_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:33311 method = PUT path = /vmms/{propolis_id}
1452024-10-17T22:41:46.460ZDEBGtest_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:33311 method = POST path = /volume/{volume_id}/remove-read-only-parent
1462024-10-17T22:41:46.460ZINFOtest_metrics (dropshot_internal): listening local_addr = 127.0.0.1:33311
1472024-10-17T22:41:46.460ZDEBGtest_metrics (dropshot_internal): successfully registered DTrace USDT probes local_addr = 127.0.0.1:33311
1482024-10-17T22:41:46.460ZINFOtest_metrics (SEC): SEC running sec_id = 017ff6a2-3b72-4882-9b0b-97601068f99c
1492024-10-17T22:41:46.460ZTRACtest_metrics (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1502024-10-17T22:41:46.460ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
1512024-10-17T22:41:46.460ZDEBGtest_metrics (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1522024-10-17T22:41:46.460ZDEBGtest_metrics (DataLoader): attempting to create built-in users
1532024-10-17T22:41:46.460ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1542024-10-17T22:41:46.460ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
1552024-10-17T22:41:46.460ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1562024-10-17T22:41:46.462ZTRACtest_metrics (dns): buffer buffer = Length: 79 (0x4f) bytes\n0000: 62 61 01 00 00 01 00 00 00 00 00 01 10 5f 63 72 ba..........._cr\n0010: 75 63 69 62 6c 65 2d 70 61 6e 74 72 79 04 5f 74 ucible-pantry._t\n0020: 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 cp.control-plane\n0030: 05 6f 78 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 .oxide.internal.\n0040: 00 21 00 01 00 00 29 04 d0 00 00 00 00 00 00 .!....)........ kind = dns peer_addr = [::1]:63247 req_id = 14f814c3-2892-47b5-ad8b-129718c7e4d6
1572024-10-17T22:41:46.462ZDEBGtest_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 25185,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_crucible-pantry._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_crucible-pantry._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 16,\n 95,\n 99,\n 114,\n 117,\n 99,\n 105,\n 98,\n 108,\n 101,\n 45,\n 112,\n 97,\n 110,\n 116,\n 114,\n 121,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63247 req_id = 14f814c3-2892-47b5-ad8b-129718c7e4d6
1582024-10-17T22:41:46.463ZERROtest_metrics (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 25185, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_crucible-pantry._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_crucible-pantry._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 16, 95, 99, 114, 117, 99, 105, 98, 108, 101, 45, 112, 97, 110, 116, 114, 121, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_crucible-pantry._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:63247 req_id = 14f814c3-2892-47b5-ad8b-129718c7e4d6
1592024-10-17T22:41:46.493ZINFOtest_metrics (DataLoader): created 5 built-in users
1602024-10-17T22:41:46.493ZTRACtest_metrics (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1612024-10-17T22:41:46.493ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
1622024-10-17T22:41:46.495ZDEBGtest_metrics (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1632024-10-17T22:41:46.495ZDEBGtest_metrics (DataLoader): attempting to create built-in roles
1642024-10-17T22:41:46.495ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1652024-10-17T22:41:46.495ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
1662024-10-17T22:41:46.496ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1672024-10-17T22:41:46.530ZINFOtest_metrics (DataLoader): created 10 built-in roles
1682024-10-17T22:41:46.530ZTRACtest_metrics (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1692024-10-17T22:41:46.530ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
1702024-10-17T22:41:46.533ZDEBGtest_metrics (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1712024-10-17T22:41:46.533ZDEBGtest_metrics (DataLoader): attempting to create built-in role assignments
1722024-10-17T22:41:46.533ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1732024-10-17T22:41:46.533ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
1742024-10-17T22:41:46.533ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1752024-10-17T22:41:47.090ZINFOtest_metrics (DataLoader): created 4 built-in role assignments
1762024-10-17T22:41:47.090ZTRACtest_metrics (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1772024-10-17T22:41:47.090ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
1782024-10-17T22:41:47.094ZDEBGtest_metrics (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1792024-10-17T22:41:47.094ZDEBGtest_metrics (DataLoader): attempting to create built-in silos
1802024-10-17T22:41:47.094ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1812024-10-17T22:41:47.095ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
1822024-10-17T22:41:47.103ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1832024-10-17T22:41:47.279ZINFOtest_metrics (DataLoader): created 2 built-in silos
1842024-10-17T22:41:47.279ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1852024-10-17T22:41:47.279ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
1862024-10-17T22:41:47.280ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1872024-10-17T22:41:47.329ZTRACtest_metrics (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1882024-10-17T22:41:47.329ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
1892024-10-17T22:41:47.331ZDEBGtest_metrics (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1902024-10-17T22:41:47.331ZDEBGtest_metrics (DataLoader): attempting to create built-in projects
1912024-10-17T22:41:47.331ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1922024-10-17T22:41:47.331ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
1932024-10-17T22:41:47.331ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1942024-10-17T22:41:47.334ZTRACtest_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }
1952024-10-17T22:41:47.334ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
1962024-10-17T22:41:47.334ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1972024-10-17T22:41:47.334ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
1982024-10-17T22:41:47.335ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1992024-10-17T22:41:47.337ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
2002024-10-17T22:41:47.337ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2012024-10-17T22:41:47.337ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2022024-10-17T22:41:47.338ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2032024-10-17T22:41:47.380ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2042024-10-17T22:41:47.383ZDEBGtest_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) } result = Ok(())
2052024-10-17T22:41:47.385ZTRACtest_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }
2062024-10-17T22:41:47.385ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
2072024-10-17T22:41:47.385ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2082024-10-17T22:41:47.385ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2092024-10-17T22:41:47.385ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2102024-10-17T22:41:47.385ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
2112024-10-17T22:41:47.386ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2122024-10-17T22:41:47.386ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2132024-10-17T22:41:47.386ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2142024-10-17T22:41:47.388ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2152024-10-17T22:41:47.392ZDEBGtest_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) } result = Ok(())
2162024-10-17T22:41:47.392ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2172024-10-17T22:41:47.392ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2182024-10-17T22:41:47.395ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2192024-10-17T22:41:47.462ZINFOtest_metrics (DataLoader): created built-in services project
2202024-10-17T22:41:47.465ZTRACtest_metrics (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2212024-10-17T22:41:47.465ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2222024-10-17T22:41:47.465ZDEBGtest_metrics (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2232024-10-17T22:41:47.465ZDEBGtest_metrics (DataLoader): attempting to create built-in VPCs
2242024-10-17T22:41:47.465ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2252024-10-17T22:41:47.465ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2262024-10-17T22:41:47.466ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2272024-10-17T22:41:47.483ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2282024-10-17T22:41:47.483ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2292024-10-17T22:41:47.484ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2302024-10-17T22:41:47.488ZTRACtest_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }
2312024-10-17T22:41:47.488ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
2322024-10-17T22:41:47.489ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2332024-10-17T22:41:47.489ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2342024-10-17T22:41:47.489ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2352024-10-17T22:41:47.495ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
2362024-10-17T22:41:47.495ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2372024-10-17T22:41:47.495ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2382024-10-17T22:41:47.495ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2392024-10-17T22:41:47.495ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
2402024-10-17T22:41:47.495ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2412024-10-17T22:41:47.495ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2422024-10-17T22:41:47.495ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2432024-10-17T22:41:47.495ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2442024-10-17T22:41:47.500ZDEBGtest_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) } result = Ok(())
2452024-10-17T22:41:47.500ZTRACtest_metrics (DataLoader): successful lookup of siloed resource "Project" using built-in user
2462024-10-17T22:41:47.500ZTRACtest_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }
2472024-10-17T22:41:47.500ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
2482024-10-17T22:41:47.500ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2492024-10-17T22:41:47.500ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2502024-10-17T22:41:47.500ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2512024-10-17T22:41:47.502ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
2522024-10-17T22:41:47.503ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2532024-10-17T22:41:47.503ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2542024-10-17T22:41:47.503ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2552024-10-17T22:41:47.511ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
2562024-10-17T22:41:47.511ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2572024-10-17T22:41:47.511ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2582024-10-17T22:41:47.511ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2592024-10-17T22:41:47.511ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2602024-10-17T22:41:47.511ZDEBGtest_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) } result = Ok(())
2612024-10-17T22:41:47.511ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2622024-10-17T22:41:47.511ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2632024-10-17T22:41:47.511ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2642024-10-17T22:41:47.557ZTRACtest_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }
2652024-10-17T22:41:47.558ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
2662024-10-17T22:41:47.558ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2672024-10-17T22:41:47.558ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2682024-10-17T22:41:47.558ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2692024-10-17T22:41:47.560ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
2702024-10-17T22:41:47.560ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2712024-10-17T22:41:47.560ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2722024-10-17T22:41:47.561ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2732024-10-17T22:41:47.563ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
2742024-10-17T22:41:47.564ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2752024-10-17T22:41:47.564ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2762024-10-17T22:41:47.564ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2772024-10-17T22:41:47.566ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2782024-10-17T22:41:47.573ZDEBGtest_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") } result = Ok(())
2792024-10-17T22:41:47.573ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2802024-10-17T22:41:47.573ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2812024-10-17T22:41:47.573ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2822024-10-17T22:41:47.600ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2832024-10-17T22:41:47.601ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2842024-10-17T22:41:47.601ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2852024-10-17T22:41:47.624ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2862024-10-17T22:41:47.624ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2872024-10-17T22:41:47.625ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2882024-10-17T22:41:47.632ZTRACtest_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }
2892024-10-17T22:41:47.632ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
2902024-10-17T22:41:47.633ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2912024-10-17T22:41:47.633ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2922024-10-17T22:41:47.633ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2932024-10-17T22:41:47.634ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
2942024-10-17T22:41:47.635ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2952024-10-17T22:41:47.635ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2962024-10-17T22:41:47.635ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2972024-10-17T22:41:47.636ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
2982024-10-17T22:41:47.637ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2992024-10-17T22:41:47.637ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3002024-10-17T22:41:47.637ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3012024-10-17T22:41:47.638ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3022024-10-17T22:41:47.644ZDEBGtest_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") } result = Ok(())
3032024-10-17T22:41:47.644ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3042024-10-17T22:41:47.644ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3052024-10-17T22:41:47.644ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3062024-10-17T22:41:47.713ZTRACtest_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) }
3072024-10-17T22:41:47.713ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
3082024-10-17T22:41:47.713ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3092024-10-17T22:41:47.713ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3102024-10-17T22:41:47.714ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3112024-10-17T22:41:47.720ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
3122024-10-17T22:41:47.720ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3132024-10-17T22:41:47.720ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3142024-10-17T22:41:47.720ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3152024-10-17T22:41:47.764ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
3162024-10-17T22:41:47.764ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3172024-10-17T22:41:47.765ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3182024-10-17T22:41:47.765ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3192024-10-17T22:41:47.768ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3202024-10-17T22:41:47.776ZDEBGtest_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
3212024-10-17T22:41:47.776ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3222024-10-17T22:41:47.776ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3232024-10-17T22:41:47.777ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3242024-10-17T22:41:47.924ZTRACtest_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) }
3252024-10-17T22:41:47.924ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
3262024-10-17T22:41:47.924ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3272024-10-17T22:41:47.924ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3282024-10-17T22:41:47.925ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3292024-10-17T22:41:47.926ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
3302024-10-17T22:41:47.926ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3312024-10-17T22:41:47.927ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3322024-10-17T22:41:47.927ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3332024-10-17T22:41:47.929ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
3342024-10-17T22:41:47.929ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3352024-10-17T22:41:47.929ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3362024-10-17T22:41:47.929ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3372024-10-17T22:41:47.932ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3382024-10-17T22:41:47.945ZDEBGtest_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
3392024-10-17T22:41:47.945ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3402024-10-17T22:41:47.945ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3412024-10-17T22:41:47.945ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3422024-10-17T22:41:48.024ZDEBGtest_metrics (DataLoader): attempting to create built-in VPC firewall rules
3432024-10-17T22:41:48.024ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3442024-10-17T22:41:48.024ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3452024-10-17T22:41:48.025ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3462024-10-17T22:41:48.036ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3472024-10-17T22:41:48.036ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3482024-10-17T22:41:48.037ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3492024-10-17T22:41:48.039ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3502024-10-17T22:41:48.039ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3512024-10-17T22:41:48.040ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3522024-10-17T22:41:48.043ZTRACtest_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
3532024-10-17T22:41:48.043ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
3542024-10-17T22:41:48.044ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3552024-10-17T22:41:48.045ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3562024-10-17T22:41:48.046ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3572024-10-17T22:41:48.048ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
3582024-10-17T22:41:48.048ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3592024-10-17T22:41:48.049ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3602024-10-17T22:41:48.049ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3612024-10-17T22:41:48.051ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
3622024-10-17T22:41:48.052ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3632024-10-17T22:41:48.052ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3642024-10-17T22:41:48.052ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3652024-10-17T22:41:48.056ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3662024-10-17T22:41:48.062ZDEBGtest_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
3672024-10-17T22:41:48.062ZTRACtest_metrics (DataLoader): successful lookup of siloed resource "Vpc" using built-in user
3682024-10-17T22:41:48.062ZTRACtest_metrics (DataLoader): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
3692024-10-17T22:41:48.062ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
3702024-10-17T22:41:48.063ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3712024-10-17T22:41:48.063ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3722024-10-17T22:41:48.063ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3732024-10-17T22:41:48.071ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
3742024-10-17T22:41:48.071ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3752024-10-17T22:41:48.071ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3762024-10-17T22:41:48.072ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3772024-10-17T22:41:48.093ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
3782024-10-17T22:41:48.095ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3792024-10-17T22:41:48.095ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3802024-10-17T22:41:48.095ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3812024-10-17T22:41:48.097ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3822024-10-17T22:41:48.109ZDEBGtest_metrics (DataLoader): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
3832024-10-17T22:41:48.112ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3842024-10-17T22:41:48.119ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3852024-10-17T22:41:48.120ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3862024-10-17T22:41:48.146ZTRACtest_metrics (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
3872024-10-17T22:41:48.146ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
3882024-10-17T22:41:48.147ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3892024-10-17T22:41:48.147ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3902024-10-17T22:41:48.147ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3912024-10-17T22:41:48.150ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
3922024-10-17T22:41:48.151ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3932024-10-17T22:41:48.151ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3942024-10-17T22:41:48.151ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3952024-10-17T22:41:48.152ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
3962024-10-17T22:41:48.153ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3972024-10-17T22:41:48.153ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3982024-10-17T22:41:48.153ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3992024-10-17T22:41:48.154ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4002024-10-17T22:41:48.160ZDEBGtest_metrics (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
4012024-10-17T22:41:48.160ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4022024-10-17T22:41:48.160ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4032024-10-17T22:41:48.160ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4042024-10-17T22:41:48.268ZINFOtest_metrics (DataLoader): created built-in VPC firewall rules
4052024-10-17T22:41:48.268ZDEBGtest_metrics (DataLoader): attempting to create built-in VPC Subnets
4062024-10-17T22:41:48.268ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4072024-10-17T22:41:48.268ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4082024-10-17T22:41:48.269ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4092024-10-17T22:41:48.281ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4102024-10-17T22:41:48.281ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4112024-10-17T22:41:48.281ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4122024-10-17T22:41:48.283ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4132024-10-17T22:41:48.283ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4142024-10-17T22:41:48.284ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4152024-10-17T22:41:48.285ZTRACtest_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
4162024-10-17T22:41:48.285ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
4172024-10-17T22:41:48.285ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4182024-10-17T22:41:48.285ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4192024-10-17T22:41:48.286ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4202024-10-17T22:41:48.287ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
4212024-10-17T22:41:48.288ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4222024-10-17T22:41:48.288ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4232024-10-17T22:41:48.288ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4242024-10-17T22:41:48.290ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
4252024-10-17T22:41:48.291ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4262024-10-17T22:41:48.291ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4272024-10-17T22:41:48.291ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4282024-10-17T22:41:48.291ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4292024-10-17T22:41:48.297ZDEBGtest_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
4302024-10-17T22:41:48.297ZTRACtest_metrics (DataLoader): successful lookup of siloed resource "Vpc" using built-in user
4312024-10-17T22:41:48.297ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4322024-10-17T22:41:48.297ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4332024-10-17T22:41:48.297ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4342024-10-17T22:41:48.321ZTRACtest_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
4352024-10-17T22:41:48.322ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
4362024-10-17T22:41:48.322ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4372024-10-17T22:41:48.322ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4382024-10-17T22:41:48.322ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4392024-10-17T22:41:48.326ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
4402024-10-17T22:41:48.326ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4412024-10-17T22:41:48.327ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4422024-10-17T22:41:48.329ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4432024-10-17T22:41:48.331ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
4442024-10-17T22:41:48.331ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4452024-10-17T22:41:48.331ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4462024-10-17T22:41:48.331ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4472024-10-17T22:41:48.333ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4482024-10-17T22:41:48.342ZDEBGtest_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
4492024-10-17T22:41:48.361ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4502024-10-17T22:41:48.361ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4512024-10-17T22:41:48.361ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4522024-10-17T22:41:48.396ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4532024-10-17T22:41:48.396ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4542024-10-17T22:41:48.396ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4552024-10-17T22:41:48.401ZTRACtest_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) }
4562024-10-17T22:41:48.401ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
4572024-10-17T22:41:48.401ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4582024-10-17T22:41:48.401ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4592024-10-17T22:41:48.402ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4602024-10-17T22:41:48.408ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
4612024-10-17T22:41:48.408ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4622024-10-17T22:41:48.408ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4632024-10-17T22:41:48.409ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4642024-10-17T22:41:48.410ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
4652024-10-17T22:41:48.410ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4662024-10-17T22:41:48.412ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4672024-10-17T22:41:48.412ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4682024-10-17T22:41:48.413ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4692024-10-17T22:41:48.420ZDEBGtest_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
4702024-10-17T22:41:48.420ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4712024-10-17T22:41:48.420ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4722024-10-17T22:41:48.420ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4732024-10-17T22:41:48.435ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4742024-10-17T22:41:48.436ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4752024-10-17T22:41:48.436ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4762024-10-17T22:41:48.438ZTRACtest_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
4772024-10-17T22:41:48.439ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
4782024-10-17T22:41:48.439ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4792024-10-17T22:41:48.439ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4802024-10-17T22:41:48.439ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4812024-10-17T22:41:48.442ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
4822024-10-17T22:41:48.444ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4832024-10-17T22:41:48.444ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4842024-10-17T22:41:48.444ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4852024-10-17T22:41:48.446ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
4862024-10-17T22:41:48.448ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4872024-10-17T22:41:48.450ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4882024-10-17T22:41:48.450ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4892024-10-17T22:41:48.450ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4902024-10-17T22:41:48.456ZDEBGtest_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
4912024-10-17T22:41:48.468ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4922024-10-17T22:41:48.468ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4932024-10-17T22:41:48.469ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4942024-10-17T22:41:48.491ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4952024-10-17T22:41:48.491ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4962024-10-17T22:41:48.492ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4972024-10-17T22:41:48.498ZTRACtest_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) }
4982024-10-17T22:41:48.498ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
4992024-10-17T22:41:48.498ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5002024-10-17T22:41:48.498ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5012024-10-17T22:41:48.502ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5022024-10-17T22:41:48.504ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
5032024-10-17T22:41:48.504ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5042024-10-17T22:41:48.504ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5052024-10-17T22:41:48.505ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5062024-10-17T22:41:48.507ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
5072024-10-17T22:41:48.507ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5082024-10-17T22:41:48.507ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5092024-10-17T22:41:48.507ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5102024-10-17T22:41:48.509ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5112024-10-17T22:41:48.542ZDEBGtest_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
5122024-10-17T22:41:48.542ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5132024-10-17T22:41:48.542ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5142024-10-17T22:41:48.543ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5152024-10-17T22:41:48.596ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5162024-10-17T22:41:48.597ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5172024-10-17T22:41:48.597ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5182024-10-17T22:41:48.600ZTRACtest_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
5192024-10-17T22:41:48.605ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
5202024-10-17T22:41:48.605ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5212024-10-17T22:41:48.605ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5222024-10-17T22:41:48.605ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5232024-10-17T22:41:48.606ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
5242024-10-17T22:41:48.606ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5252024-10-17T22:41:48.606ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5262024-10-17T22:41:48.606ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5272024-10-17T22:41:48.615ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
5282024-10-17T22:41:48.615ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5292024-10-17T22:41:48.615ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5302024-10-17T22:41:48.616ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5312024-10-17T22:41:48.618ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5322024-10-17T22:41:48.625ZDEBGtest_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
5332024-10-17T22:41:48.633ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5342024-10-17T22:41:48.633ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5352024-10-17T22:41:48.634ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5362024-10-17T22:41:48.661ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5372024-10-17T22:41:48.662ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5382024-10-17T22:41:48.662ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5392024-10-17T22:41:48.668ZTRACtest_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) }
5402024-10-17T22:41:48.668ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
5412024-10-17T22:41:48.668ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5422024-10-17T22:41:48.669ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5432024-10-17T22:41:48.669ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5442024-10-17T22:41:48.672ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
5452024-10-17T22:41:48.672ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5462024-10-17T22:41:48.672ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5472024-10-17T22:41:48.674ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5482024-10-17T22:41:48.678ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
5492024-10-17T22:41:48.678ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5502024-10-17T22:41:48.678ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5512024-10-17T22:41:48.679ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5522024-10-17T22:41:48.685ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5532024-10-17T22:41:48.698ZDEBGtest_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
5542024-10-17T22:41:48.698ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5552024-10-17T22:41:48.700ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5562024-10-17T22:41:48.700ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5572024-10-17T22:41:48.714ZINFOtest_metrics (DataLoader): created built-in services vpc subnets
5582024-10-17T22:41:48.714ZINFOtest_metrics (DataLoader): created built-in services vpc
5592024-10-17T22:41:48.714ZTRACtest_metrics (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5602024-10-17T22:41:48.714ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5612024-10-17T22:41:48.717ZDEBGtest_metrics (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5622024-10-17T22:41:48.725ZDEBGtest_metrics (DataLoader): attempting to create silo users
5632024-10-17T22:41:48.725ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5642024-10-17T22:41:48.725ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5652024-10-17T22:41:48.726ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5662024-10-17T22:41:48.761ZINFOtest_metrics (DataLoader): created 2 silo users
5672024-10-17T22:41:48.761ZDEBGtest_metrics (DataLoader): attempting to create silo user role assignments
5682024-10-17T22:41:48.761ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5692024-10-17T22:41:48.761ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5702024-10-17T22:41:48.762ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5712024-10-17T22:41:48.769ZINFOtest_metrics (DataLoader): created 2 silo user role assignments
5722024-10-17T22:41:48.769ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5732024-10-17T22:41:48.769ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5742024-10-17T22:41:48.773ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5752024-10-17T22:41:48.783ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5762024-10-17T22:41:48.783ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5772024-10-17T22:41:48.784ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5782024-10-17T22:41:48.800ZTRACtest_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = IpPoolList
5792024-10-17T22:41:48.800ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
5802024-10-17T22:41:48.800ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5812024-10-17T22:41:48.800ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5822024-10-17T22:41:48.801ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5832024-10-17T22:41:48.802ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5842024-10-17T22:41:48.805ZDEBGtest_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = IpPoolList result = Ok(())
5852024-10-17T22:41:48.805ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5862024-10-17T22:41:48.805ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5872024-10-17T22:41:48.806ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5882024-10-17T22:41:48.834ZTRACtest_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = IpPoolList
5892024-10-17T22:41:48.834ZTRACtest_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
5902024-10-17T22:41:48.835ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5912024-10-17T22:41:48.835ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5922024-10-17T22:41:48.835ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5932024-10-17T22:41:48.837ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5942024-10-17T22:41:48.839ZDEBGtest_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = IpPoolList result = Ok(())
5952024-10-17T22:41:48.839ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5962024-10-17T22:41:48.839ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5972024-10-17T22:41:48.842ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5982024-10-17T22:41:49.132ZTRACtest_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5992024-10-17T22:41:49.133ZDEBGtest_metrics (DataLoader): roles roles = RoleSet { roles: {} }
6002024-10-17T22:41:49.134ZDEBGtest_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
6012024-10-17T22:41:49.160ZINFOtest_metrics (nexus): populate complete
6022024-10-17T22:41:49.160ZINFOtest_metrics (nexus): activating background tasks
6032024-10-17T22:41:49.172ZDEBGtest_metrics (init_with_steps): Running step start_sled1
6042024-10-17T22:41:49.172ZINFOtest_metrics (omicron_sled_agent::sim::Server): setting up sled agent server sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6052024-10-17T22:41:49.260ZINFOtest_metrics (SledAgent): created simulated sled agent server = b6d65341-167c-41df-9b5c-41cded99c229 sim_mode = Explicit sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6062024-10-17T22:41:49.263ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40953 method = POST path = /boot-disk/{boot_disk}/os/write sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6072024-10-17T22:41:49.267ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40953 method = GET path = /boot-disk/{boot_disk}/os/write/status sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6082024-10-17T22:41:49.269ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40953 method = DELETE path = /boot-disk/{boot_disk}/os/write/status/{update_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6092024-10-17T22:41:49.269ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40953 method = GET path = /bootstore/status sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6102024-10-17T22:41:49.269ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40953 method = POST path = /cockroachdb sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6112024-10-17T22:41:49.269ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40953 method = GET path = /datasets sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6122024-10-17T22:41:49.269ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40953 method = PUT path = /datasets sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6132024-10-17T22:41:49.269ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40953 method = PUT path = /disks/{disk_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6142024-10-17T22:41:49.269ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40953 method = POST path = /disks/{disk_id}/poke sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6152024-10-17T22:41:49.269ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40953 method = PUT path = /eip-gateways sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6162024-10-17T22:41:49.269ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40953 method = GET path = /inventory sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6172024-10-17T22:41:49.269ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40953 method = GET path = /network-bootstore-config sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6182024-10-17T22:41:49.269ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40953 method = PUT path = /network-bootstore-config sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6192024-10-17T22:41:49.269ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40953 method = GET path = /omicron-physical-disks sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6202024-10-17T22:41:49.269ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40953 method = PUT path = /omicron-physical-disks sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6212024-10-17T22:41:49.269ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40953 method = PUT path = /omicron-zones sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6222024-10-17T22:41:49.269ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40953 method = GET path = /sled-identifiers sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6232024-10-17T22:41:49.270ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40953 method = GET path = /sled-role sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6242024-10-17T22:41:49.270ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40953 method = PUT path = /sleds sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6252024-10-17T22:41:49.270ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40953 method = POST path = /switch-ports sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6262024-10-17T22:41:49.270ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40953 method = GET path = /timesync sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6272024-10-17T22:41:49.270ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40953 method = POST path = /update sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6282024-10-17T22:41:49.270ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40953 method = DELETE path = /v2p sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6292024-10-17T22:41:49.270ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40953 method = GET path = /v2p sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6302024-10-17T22:41:49.270ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40953 method = PUT path = /v2p sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6312024-10-17T22:41:49.270ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40953 method = DELETE path = /vmms/{propolis_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6322024-10-17T22:41:49.270ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40953 method = PUT path = /vmms/{propolis_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6332024-10-17T22:41:49.270ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40953 method = POST path = /vmms/{propolis_id}/disks/{disk_id}/snapshot sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6342024-10-17T22:41:49.270ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40953 method = DELETE path = /vmms/{propolis_id}/external-ip sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6352024-10-17T22:41:49.270ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40953 method = PUT path = /vmms/{propolis_id}/external-ip sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6362024-10-17T22:41:49.270ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40953 method = POST path = /vmms/{propolis_id}/poke sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6372024-10-17T22:41:49.270ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40953 method = POST path = /vmms/{propolis_id}/poke-single-step sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6382024-10-17T22:41:49.270ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40953 method = POST path = /vmms/{propolis_id}/sim-migration-source sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6392024-10-17T22:41:49.270ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40953 method = GET path = /vmms/{propolis_id}/state sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6402024-10-17T22:41:49.270ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40953 method = PUT path = /vmms/{propolis_id}/state sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6412024-10-17T22:41:49.271ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40953 method = PUT path = /vpc/{vpc_id}/firewall/rules sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6422024-10-17T22:41:49.271ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40953 method = GET path = /vpc-routes sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6432024-10-17T22:41:49.271ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40953 method = PUT path = /vpc-routes sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6442024-10-17T22:41:49.271ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40953 method = GET path = /zones sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6452024-10-17T22:41:49.271ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40953 method = POST path = /zones/bundle-cleanup sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6462024-10-17T22:41:49.271ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40953 method = GET path = /zones/bundle-cleanup/context sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6472024-10-17T22:41:49.271ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40953 method = PUT path = /zones/bundle-cleanup/context sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6482024-10-17T22:41:49.271ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40953 method = GET path = /zones/bundle-cleanup/utilization sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6492024-10-17T22:41:49.271ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40953 method = GET path = /zones/bundles sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6502024-10-17T22:41:49.271ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40953 method = GET path = /zones/bundles/{zone_name} sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6512024-10-17T22:41:49.271ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40953 method = POST path = /zones/bundles/{zone_name} sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6522024-10-17T22:41:49.271ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40953 method = DELETE path = /zones/bundles/{zone_name}/{bundle_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6532024-10-17T22:41:49.271ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40953 method = GET path = /zones/bundles/{zone_name}/{bundle_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6542024-10-17T22:41:49.271ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40953 method = GET path = /zpools sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6552024-10-17T22:41:49.271ZINFOtest_metrics (dropshot): listening local_addr = [::1]:40953 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6562024-10-17T22:41:49.271ZDEBGtest_metrics (dropshot): successfully registered DTrace USDT probes local_addr = [::1]:40953 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6572024-10-17T22:41:49.271ZDEBGtest_metrics (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 1 reason = Timeout
6582024-10-17T22:41:49.271ZDEBGtest_metrics (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
6592024-10-17T22:41:49.271ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
6602024-10-17T22:41:49.272ZDEBGtest_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6612024-10-17T22:41:49.272ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6622024-10-17T22:41:49.272ZDEBGtest_metrics (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 1 reason = Timeout
6632024-10-17T22:41:49.272ZTRACtest_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal
6642024-10-17T22:41:49.272ZDEBGtest_metrics (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 1 reason = Timeout
6652024-10-17T22:41:49.273ZWARNtest_metrics (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_internal dns_group = internal reason = no config nor servers
6662024-10-17T22:41:49.273ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 148.014\u{b5}s iteration = 1 status = {"error":"no config nor servers"}
6672024-10-17T22:41:49.273ZDEBGtest_metrics (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 1 reason = Timeout
6682024-10-17T22:41:49.273ZDEBGtest_metrics (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
6692024-10-17T22:41:49.273ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
6702024-10-17T22:41:49.273ZDEBGtest_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6712024-10-17T22:41:49.274ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6722024-10-17T22:41:49.274ZDEBGtest_metrics (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 1 reason = Timeout
6732024-10-17T22:41:49.274ZTRACtest_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal
6742024-10-17T22:41:49.275ZDEBGtest_metrics (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 1 reason = Timeout
6752024-10-17T22:41:49.275ZWARNtest_metrics (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_external dns_group = external reason = no config nor servers
6762024-10-17T22:41:49.275ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 264.406\u{b5}s iteration = 1 status = {"error":"no config nor servers"}
6772024-10-17T22:41:49.275ZDEBGtest_metrics (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 1 reason = Timeout
6782024-10-17T22:41:49.276ZINFOtest_metrics (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-10-17 22:31:49.275720866 UTC
6792024-10-17T22:41:49.276ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database
6802024-10-17T22:41:49.276ZDEBGtest_metrics (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
6812024-10-17T22:41:49.276ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
6822024-10-17T22:41:49.276ZDEBGtest_metrics (BackgroundTasks): activating background_task = external_endpoints iteration = 1 reason = Timeout
6832024-10-17T22:41:49.276ZTRACtest_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet
6842024-10-17T22:41:49.277ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = external_endpoints resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
6852024-10-17T22:41:49.277ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
6862024-10-17T22:41:49.277ZDEBGtest_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6872024-10-17T22:41:49.278ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6882024-10-17T22:41:49.278ZDEBGtest_metrics (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 1 reason = Timeout
6892024-10-17T22:41:49.278ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database
6902024-10-17T22:41:49.278ZDEBGtest_metrics (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
6912024-10-17T22:41:49.279ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
6922024-10-17T22:41:49.279ZDEBGtest_metrics (BackgroundTasks): activating background_task = bfd_manager iteration = 1 reason = Timeout
6932024-10-17T22:41:49.279ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database
6942024-10-17T22:41:49.279ZDEBGtest_metrics (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
6952024-10-17T22:41:49.280ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
6962024-10-17T22:41:49.280ZDEBGtest_metrics (BackgroundTasks): activating background_task = phantom_disks iteration = 1 reason = Timeout
6972024-10-17T22:41:49.280ZDEBGtest_metrics (BackgroundTasks): activating background_task = blueprint_loader iteration = 1 reason = Timeout
6982024-10-17T22:41:49.281ZTRACtest_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig
6992024-10-17T22:41:49.281ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = blueprint_loader resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
7002024-10-17T22:41:49.281ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database
7012024-10-17T22:41:49.281ZDEBGtest_metrics (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7022024-10-17T22:41:49.281ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7032024-10-17T22:41:49.281ZDEBGtest_metrics (BackgroundTasks): activating background_task = blueprint_executor iteration = 1 reason = Timeout
7042024-10-17T22:41:49.281ZWARNtest_metrics (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor reason = no blueprint
7052024-10-17T22:41:49.281ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 32.7\u{b5}s iteration = 1 status = {"error":"no blueprint"}
7062024-10-17T22:41:49.281ZDEBGtest_metrics (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 1 reason = Timeout
7072024-10-17T22:41:49.281ZWARNtest_metrics (BackgroundTasks): Blueprint execution: skipped background_task = crdb_node_id_collector reason = no blueprint
7082024-10-17T22:41:49.282ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 14.89\u{b5}s iteration = 1 status = {"error":"no blueprint"}
7092024-10-17T22:41:49.282ZDEBGtest_metrics (omicron_sled_agent::sim::Server): contacting server nexus sled_id = b6d65341-167c-41df-9b5c-41cded99c229
7102024-10-17T22:41:49.282ZDEBGtest_metrics (NexusClient): client request body = Some(Body) method = POST sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = http://127.0.0.1:33311/sled-agents/b6d65341-167c-41df-9b5c-41cded99c229
7112024-10-17T22:41:49.283ZINFOtest_metrics (dropshot_internal): accepted connection local_addr = 127.0.0.1:33311 remote_addr = 127.0.0.1:36380
7122024-10-17T22:41:49.285ZDEBGtest_metrics (BackgroundTasks): activating background_task = inventory_collection iteration = 1 reason = Timeout
7132024-10-17T22:41:49.285ZTRACtest_metrics (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
7142024-10-17T22:41:49.285ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
7152024-10-17T22:41:49.285ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
7162024-10-17T22:41:49.285ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
7172024-10-17T22:41:49.285ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
7182024-10-17T22:41:49.285ZDEBGtest_metrics (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 1 reason = Timeout
7192024-10-17T22:41:49.286ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 20.881\u{b5}s iteration = 1 status = {"error":"task disabled"}
7202024-10-17T22:41:49.286ZDEBGtest_metrics (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 1 reason = Timeout
7212024-10-17T22:41:49.286ZINFOtest_metrics (BackgroundTasks): Decommissioned Disk Cleaner disabled background_task = decommissioned_disk_cleaner
7222024-10-17T22:41:49.286ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 27.821\u{b5}s iteration = 1 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
7232024-10-17T22:41:49.286ZDEBGtest_metrics (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 1 reason = Timeout
7242024-10-17T22:41:49.286ZTRACtest_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory
7252024-10-17T22:41:49.286ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_zone_nat_tracker resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
7262024-10-17T22:41:49.286ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database
7272024-10-17T22:41:49.286ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7282024-10-17T22:41:49.286ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7292024-10-17T22:41:49.286ZDEBGtest_metrics (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 1 reason = Timeout
7302024-10-17T22:41:49.286ZTRACtest_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet
7312024-10-17T22:41:49.286ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = switch_port_config_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
7322024-10-17T22:41:49.286ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database
7332024-10-17T22:41:49.286ZDEBGtest_metrics (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7342024-10-17T22:41:49.287ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7352024-10-17T22:41:49.288ZDEBGtest_metrics (BackgroundTasks): activating background_task = v2p_manager iteration = 1 reason = Timeout
7362024-10-17T22:41:49.288ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
7372024-10-17T22:41:49.288ZDEBGtest_metrics (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7382024-10-17T22:41:49.288ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7392024-10-17T22:41:49.288ZDEBGtest_metrics (BackgroundTasks): activating background_task = region_replacement iteration = 1 reason = Timeout
7402024-10-17T22:41:49.289ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database
7412024-10-17T22:41:49.289ZDEBGtest_metrics (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
7422024-10-17T22:41:49.289ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
7432024-10-17T22:41:49.289ZDEBGtest_metrics (BackgroundTasks): activating background_task = region_replacement_driver iteration = 1 reason = Timeout
7442024-10-17T22:41:49.290ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database
7452024-10-17T22:41:49.290ZDEBGtest_metrics (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
7462024-10-17T22:41:49.290ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7472024-10-17T22:41:49.291ZDEBGtest_metrics (BackgroundTasks): activating background_task = instance_watcher iteration = 1 reason = Timeout
7482024-10-17T22:41:49.291ZTRACtest_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet
7492024-10-17T22:41:49.291ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = instance_watcher resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
7502024-10-17T22:41:49.291ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database
7512024-10-17T22:41:49.291ZDEBGtest_metrics (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
7522024-10-17T22:41:49.292ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
7532024-10-17T22:41:49.292ZDEBGtest_metrics (BackgroundTasks): activating background_task = instance_updater iteration = 1 reason = Timeout
7542024-10-17T22:41:49.292ZINFOtest_metrics (BackgroundTasks): background instance updater explicitly disabled background_task = instance_updater
7552024-10-17T22:41:49.292ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = instance_updater elapsed = 283.377\u{b5}s iteration = 1 status = {"destroyed_active_vmms":0,"disabled":true,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
7562024-10-17T22:41:49.292ZDEBGtest_metrics (BackgroundTasks): activating background_task = instance_reincarnation iteration = 1 reason = Timeout
7572024-10-17T22:41:49.293ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database
7582024-10-17T22:41:49.294ZDEBGtest_metrics (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
7592024-10-17T22:41:49.294ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
7602024-10-17T22:41:49.294ZDEBGtest_metrics (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 1 reason = Timeout
7612024-10-17T22:41:49.294ZDEBGtest_metrics (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
7622024-10-17T22:41:49.294ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
7632024-10-17T22:41:49.294ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7642024-10-17T22:41:49.294ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7652024-10-17T22:41:49.294ZDEBGtest_metrics (BackgroundTasks): activating background_task = vpc_route_manager iteration = 1 reason = Timeout
7662024-10-17T22:41:49.294ZINFOtest_metrics (BackgroundTasks): VPC route manager running background_task = vpc_route_manager
7672024-10-17T22:41:49.295ZTRACtest_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet
7682024-10-17T22:41:49.295ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = vpc_route_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
7692024-10-17T22:41:49.295ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
7702024-10-17T22:41:49.295ZDEBGtest_metrics (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7712024-10-17T22:41:49.296ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7722024-10-17T22:41:49.296ZDEBGtest_metrics (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 1 reason = Timeout
7732024-10-17T22:41:49.296ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database
7742024-10-17T22:41:49.297ZDEBGtest_metrics (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
7752024-10-17T22:41:49.297ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
7762024-10-17T22:41:49.297ZDEBGtest_metrics (BackgroundTasks): activating background_task = saga_recovery iteration = 1 reason = Timeout
7772024-10-17T22:41:49.297ZDEBGtest_metrics (SagaRecoverer): listing candidate sagas for recovery
7782024-10-17T22:41:49.297ZTRACtest_metrics (SagaRecoverer): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000003a8a, .. }) resource = Database
7792024-10-17T22:41:49.297ZDEBGtest_metrics (SagaRecoverer): roles roles = RoleSet { roles: {} }
7802024-10-17T22:41:49.298ZDEBGtest_metrics (SagaRecoverer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000003a8a, .. }) resource = Database result = Ok(())
7812024-10-17T22:41:49.298ZDEBGtest_metrics (BackgroundTasks): activating background_task = lookup_region_port iteration = 1 reason = Timeout
7822024-10-17T22:41:49.298ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database
7832024-10-17T22:41:49.298ZDEBGtest_metrics (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
7842024-10-17T22:41:49.299ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
7852024-10-17T22:41:49.300ZDEBGtest_metrics (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 1 reason = Timeout
7862024-10-17T22:41:49.300ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 339.538\u{b5}s iteration = 1 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
7872024-10-17T22:41:49.300ZDEBGtest_metrics (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 1 reason = Timeout
7882024-10-17T22:41:49.300ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 127.312\u{b5}s iteration = 1 status = {"errors":[],"garbage_collect_requested":[]}
7892024-10-17T22:41:49.301ZDEBGtest_metrics (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 1 reason = Timeout
7902024-10-17T22:41:49.301ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 148.084\u{b5}s iteration = 1 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
7912024-10-17T22:41:49.301ZDEBGtest_metrics (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 1 reason = Timeout
7922024-10-17T22:41:49.301ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 126.803\u{b5}s iteration = 1 status = {"errors":[],"records_set_to_done":[]}
7932024-10-17T22:41:49.301ZTRACtest_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig
7942024-10-17T22:41:49.302ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_internal dns_group = internal resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
7952024-10-17T22:41:49.302ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
7962024-10-17T22:41:49.302ZDEBGtest_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
7972024-10-17T22:41:49.302ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
7982024-10-17T22:41:49.304ZTRACtest_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig
7992024-10-17T22:41:49.304ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_external dns_group = external resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
8002024-10-17T22:41:49.304ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
8012024-10-17T22:41:49.304ZDEBGtest_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
8022024-10-17T22:41:49.305ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
8032024-10-17T22:41:49.341ZTRACtest_metrics (dropshot_internal): incoming request local_addr = 127.0.0.1:33311 method = POST remote_addr = 127.0.0.1:36380 req_id = 2edce9e7-cb53-41bb-9d59-cfde809114b9 uri = /sled-agents/b6d65341-167c-41df-9b5c-41cded99c229
8042024-10-17T22:41:49.344ZINFOtest_metrics (nexus): registered sled agent sled_uuid = b6d65341-167c-41df-9b5c-41cded99c229
8052024-10-17T22:41:49.344ZTRACtest_metrics (dns): buffer buffer = Length: 75 (0x4b) bytes\n0000: d5 a5 01 00 00 01 00 00 00 00 00 01 0c 5f 6e 61 ............._na\n0010: 6d 65 73 65 72 76 69 63 65 04 5f 74 63 70 0d 63 meservice._tcp.c\n0020: 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 ontrol-plane.oxi\n0030: 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 de.internal..!..\n0040: 00 00 29 04 d0 00 00 00 00 00 00 ..)........ kind = dns peer_addr = [::1]:55978 req_id = 8cabc0dd-6792-4eae-a3da-a73cd067cc99
8062024-10-17T22:41:49.345ZDEBGtest_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 54693,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_nameservice._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_nameservice._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 12,\n 95,\n 110,\n 97,\n 109,\n 101,\n 115,\n 101,\n 114,\n 118,\n 105,\n 99,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:55978 req_id = 8cabc0dd-6792-4eae-a3da-a73cd067cc99
8072024-10-17T22:41:49.346ZERROtest_metrics (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 54693, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_nameservice._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 12, 95, 110, 97, 109, 101, 115, 101, 114, 118, 105, 99, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_nameservice._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:55978 req_id = 8cabc0dd-6792-4eae-a3da-a73cd067cc99
8082024-10-17T22:41:49.347ZTRACtest_metrics (dns): buffer buffer = Length: 76 (0x4c) bytes\n0000: bb 17 01 00 00 01 00 00 00 00 00 01 0d 5f 65 78 ............._ex\n0010: 74 65 72 6e 61 6c 2d 64 6e 73 04 5f 74 63 70 0d ternal-dns._tcp.\n0020: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0030: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 21 00 ide.internal..!.\n0040: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:60752 req_id = ca4ad52e-db03-462b-a879-0ad3217b92df
8092024-10-17T22:41:49.347ZDEBGtest_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 47895,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_external-dns._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_external-dns._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 13,\n 95,\n 101,\n 120,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 45,\n 100,\n 110,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60752 req_id = ca4ad52e-db03-462b-a879-0ad3217b92df
8102024-10-17T22:41:49.347ZERROtest_metrics (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 47895, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_external-dns._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 13, 95, 101, 120, 116, 101, 114, 110, 97, 108, 45, 100, 110, 115, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_external-dns._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:60752 req_id = ca4ad52e-db03-462b-a879-0ad3217b92df
8112024-10-17T22:41:49.349ZWARNtest_metrics (BackgroundTasks): failed to lookup DNS servers background_task = dns_servers_internal dns_group = internal error = no record found for Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
8122024-10-17T22:41:49.350ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 81.138382ms iteration = 1 status = {"error":"failed to read list of DNS servers: no record found for Query { name: Name(\\"_nameservice._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }"}
8132024-10-17T22:41:49.350ZWARNtest_metrics (BackgroundTasks): failed to lookup DNS servers background_task = dns_servers_external dns_group = external error = no record found for Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
8142024-10-17T22:41:49.350ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 75.446164ms iteration = 1 status = {"error":"failed to read list of DNS servers: no record found for Query { name: Name(\\"_external-dns._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }"}
8152024-10-17T22:41:49.356ZINFOtest_metrics (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-10-17 22:31:49.275720866 UTC pruned = {}
8162024-10-17T22:41:49.356ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 80.749762ms iteration = 1 status = {"expiration":"2024-10-17T22:31:49.275720866Z","pruned":[]}
8172024-10-17T22:41:49.367ZTRACtest_metrics (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
8182024-10-17T22:41:49.368ZTRACtest_metrics (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: d4 92 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 ............._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:52849 req_id = 49b91cee-87df-4e3c-8fe0-62d37bd05197
8192024-10-17T22:41:49.368ZDEBGtest_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 54418,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52849 req_id = 49b91cee-87df-4e3c-8fe0-62d37bd05197
8202024-10-17T22:41:49.368ZERROtest_metrics (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 54418, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_dendrite._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 9, 95, 100, 101, 110, 100, 114, 105, 116, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_dendrite._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:52849 req_id = 49b91cee-87df-4e3c-8fe0-62d37bd05197
8212024-10-17T22:41:49.369ZERROtest_metrics (BackgroundTasks): failed to resolve addresses for Dendrite services background_task = nat_v4_garbage_collector error = no record found for Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
8222024-10-17T22:41:49.369ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 91.021689ms iteration = 1 status = {"error":"failed to resolve addresses for Dendrite services: no record found for Query { name: Name(\\"_dendrite._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }"}
8232024-10-17T22:41:49.383ZTRACtest_metrics (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
8242024-10-17T22:41:49.383ZDEBGtest_metrics (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8252024-10-17T22:41:49.396ZDEBGtest_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
8262024-10-17T22:41:49.396ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database
8272024-10-17T22:41:49.400ZDEBGtest_metrics (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8282024-10-17T22:41:49.400ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8292024-10-17T22:41:49.400ZTRACtest_metrics (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 29 51 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 )Q..........._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:63770 req_id = c25dc4ca-5595-4242-9e5c-21c2891ed388
8302024-10-17T22:41:49.400ZDEBGtest_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 10577,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63770 req_id = c25dc4ca-5595-4242-9e5c-21c2891ed388
8312024-10-17T22:41:49.400ZERROtest_metrics (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 10577, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_dendrite._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 9, 95, 100, 101, 110, 100, 114, 105, 116, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_dendrite._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:63770 req_id = c25dc4ca-5595-4242-9e5c-21c2891ed388
8322024-10-17T22:41:49.407ZERROtest_metrics (BackgroundTasks): failed to resolve addresses for Dendrite services background_task = bfd_manager error = no record found for Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
8332024-10-17T22:41:49.408ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 127.947807ms iteration = 1 status = {"error":"failed to resolve addresses for Dendrite services: no record found for Query { name: Name(\\"_dendrite._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }"}
8342024-10-17T22:41:49.461ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 180.944822ms iteration = 1 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
8352024-10-17T22:41:49.468ZDEBGtest_metrics (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8362024-10-17T22:41:49.473ZDEBGtest_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
8372024-10-17T22:41:49.473ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database
8382024-10-17T22:41:49.473ZDEBGtest_metrics (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
8392024-10-17T22:41:49.473ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
8402024-10-17T22:41:49.520ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
8412024-10-17T22:41:49.520ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8422024-10-17T22:41:49.520ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8432024-10-17T22:41:49.523ZDEBGtest_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8442024-10-17T22:41:49.526ZDEBGtest_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
8452024-10-17T22:41:49.526ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
8462024-10-17T22:41:49.526ZDEBGtest_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
8472024-10-17T22:41:49.526ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
8482024-10-17T22:41:49.527ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8492024-10-17T22:41:49.541ZDEBGtest_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
8502024-10-17T22:41:49.542ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database
8512024-10-17T22:41:49.542ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8522024-10-17T22:41:49.545ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8532024-10-17T22:41:49.555ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8542024-10-17T22:41:49.563ZDEBGtest_metrics (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
8552024-10-17T22:41:49.563ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
8562024-10-17T22:41:49.563ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
8572024-10-17T22:41:49.563ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
8582024-10-17T22:41:49.607ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 309.112045ms iteration = 1 status = {"errors":[],"found_port_ok":[]}
8592024-10-17T22:41:49.630ZDEBGtest_metrics (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8602024-10-17T22:41:49.645ZDEBGtest_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
8612024-10-17T22:41:49.645ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database
8622024-10-17T22:41:49.645ZDEBGtest_metrics (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8632024-10-17T22:41:49.645ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8642024-10-17T22:41:49.646ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 360.967292ms iteration = 1 status = {}
8652024-10-17T22:41:49.647ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
8662024-10-17T22:41:49.647ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8672024-10-17T22:41:49.666ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8682024-10-17T22:41:49.667ZDEBGtest_metrics (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8692024-10-17T22:41:49.690ZDEBGtest_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
8702024-10-17T22:41:49.690ZTRACtest_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet
8712024-10-17T22:41:49.690ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = vpc_route_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
8722024-10-17T22:41:49.690ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
8732024-10-17T22:41:49.690ZDEBGtest_metrics (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8742024-10-17T22:41:49.691ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8752024-10-17T22:41:49.698ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database
8762024-10-17T22:41:49.698ZDEBGtest_metrics (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
8772024-10-17T22:41:49.698ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
8782024-10-17T22:41:49.698ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database
8792024-10-17T22:41:49.698ZDEBGtest_metrics (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
8802024-10-17T22:41:49.700ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
8812024-10-17T22:41:49.705ZINFOtest_metrics (SagaRecoverer): listed in-progress sagas count = 0
8822024-10-17T22:41:49.705ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = saga_recovery elapsed = 406.709082ms iteration = 1 status = {"last_pass":{"Success":{"nfailed":0,"nfound":0,"nrecovered":0,"nremoved":0,"nskipped":0}},"ntotal_failures":0,"ntotal_finished":0,"ntotal_recovered":0,"ntotal_sec_errors_bad_state":0,"ntotal_sec_errors_missing":0,"ntotal_started":0,"recent_failures":[],"recent_recoveries":[]}
8832024-10-17T22:41:49.727ZDEBGtest_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8842024-10-17T22:41:49.734ZDEBGtest_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
8852024-10-17T22:41:49.739ZTRACtest_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig
8862024-10-17T22:41:49.739ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = external_endpoints resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
8872024-10-17T22:41:49.739ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
8882024-10-17T22:41:49.739ZDEBGtest_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
8892024-10-17T22:41:49.739ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
8902024-10-17T22:41:49.757ZDEBGtest_metrics (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
8912024-10-17T22:41:49.758ZINFOtest_metrics (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
8922024-10-17T22:41:49.758ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 461.136201ms iteration = 1 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
8932024-10-17T22:41:49.760ZTRACtest_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
8942024-10-17T22:41:49.761ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
8952024-10-17T22:41:49.761ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
8962024-10-17T22:41:49.761ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8972024-10-17T22:41:49.761ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8982024-10-17T22:41:49.764ZDEBGtest_metrics (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8992024-10-17T22:41:49.770ZDEBGtest_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
9002024-10-17T22:41:49.771ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
9012024-10-17T22:41:49.771ZDEBGtest_metrics (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9022024-10-17T22:41:49.771ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9032024-10-17T22:41:49.775ZINFOtest_metrics (BackgroundTasks): all instance checks complete background_task = instance_watcher pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
9042024-10-17T22:41:49.775ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 484.636186ms iteration = 1 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
9052024-10-17T22:41:49.796ZDEBGtest_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9062024-10-17T22:41:49.808ZDEBGtest_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
9072024-10-17T22:41:49.808ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
9082024-10-17T22:41:49.808ZDEBGtest_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
9092024-10-17T22:41:49.820ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
9102024-10-17T22:41:49.821ZDEBGtest_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9112024-10-17T22:41:49.842ZDEBGtest_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
9122024-10-17T22:41:49.842ZWARNtest_metrics (BackgroundTasks): inventory collection is None background_task = service_zone_nat_tracker
9132024-10-17T22:41:49.842ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 558.297498ms iteration = 1 status = {"error":"inventory collection is None"}
9142024-10-17T22:41:49.842ZDEBGtest_metrics (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = []
9152024-10-17T22:41:49.842ZTRACtest_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
9162024-10-17T22:41:49.843ZWARNtest_metrics (BackgroundTasks): failed to read target blueprint background_task = blueprint_loader error = Internal Error: no target blueprint set
9172024-10-17T22:41:49.843ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 562.899638ms iteration = 1 status = {"error":"failed to read target blueprint: Internal Error: no target blueprint set"}
9182024-10-17T22:41:49.843ZTRACtest_metrics (BackgroundTasks): no more instances in need of reincarnation background_task = instance_reincarnation reincarnation_reason = instance failed total_found = 0
9192024-10-17T22:41:49.843ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database
9202024-10-17T22:41:49.843ZDEBGtest_metrics (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
9212024-10-17T22:41:49.844ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
9222024-10-17T22:41:49.844ZTRACtest_metrics (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 81 1e 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:61951 req_id = 0944669b-5f4f-436c-93fe-ebf4374e5ab0
9232024-10-17T22:41:49.844ZDEBGtest_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 33054,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:61951 req_id = 0944669b-5f4f-436c-93fe-ebf4374e5ab0
9242024-10-17T22:41:49.844ZERROtest_metrics (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 33054, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_mgs._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 4, 95, 109, 103, 115, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_mgs._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:61951 req_id = 0944669b-5f4f-436c-93fe-ebf4374e5ab0
9252024-10-17T22:41:49.845ZWARNtest_metrics (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: looking up MGS addresses: no record found for Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
9262024-10-17T22:41:49.845ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 561.579397ms iteration = 1 status = {"error":"failed to collect inventory: looking up MGS addresses: no record found for Query { name: Name(\\"_mgs._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }"}
9272024-10-17T22:41:49.885ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
9282024-10-17T22:41:49.886ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
9292024-10-17T22:41:49.886ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
9302024-10-17T22:41:49.886ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
9312024-10-17T22:41:49.905ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = region_replacement elapsed = 616.315573ms iteration = 1 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
9322024-10-17T22:41:49.905ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
9332024-10-17T22:41:49.905ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
9342024-10-17T22:41:49.905ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
9352024-10-17T22:41:49.906ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
9362024-10-17T22:41:49.923ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 633.041225ms iteration = 1 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
9372024-10-17T22:41:49.944ZWARNtest_metrics (BackgroundTasks): failed to read DNS config background_task = dns_config_external dns_group = external error = Internal Error: expected exactly one latest version for DNS group External, found 0
9382024-10-17T22:41:49.944ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 671.333537ms iteration = 1 status = {"error":"failed to read DNS config: Internal Error: expected exactly one latest version for DNS group External, found 0"}
9392024-10-17T22:41:49.960ZDEBGtest_metrics (BackgroundTasks): activating background_task = inventory_collection iteration = 2 reason = Signaled
9402024-10-17T22:41:49.961ZTRACtest_metrics (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
9412024-10-17T22:41:49.961ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
9422024-10-17T22:41:49.961ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
9432024-10-17T22:41:49.961ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
9442024-10-17T22:41:49.961ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
9452024-10-17T22:41:49.961ZINFOtest_metrics (dropshot_internal): request completed latency_us = 620203 local_addr = 127.0.0.1:33311 method = POST remote_addr = 127.0.0.1:36380 req_id = 2edce9e7-cb53-41bb-9d59-cfde809114b9 response_code = 204 uri = /sled-agents/b6d65341-167c-41df-9b5c-41cded99c229
9462024-10-17T22:41:49.962ZDEBGtest_metrics (NexusClient): client response result = Ok(Response { url: "http://127.0.0.1:33311/sled-agents/b6d65341-167c-41df-9b5c-41cded99c229", status: 204, headers: {"x-request-id": "2edce9e7-cb53-41bb-9d59-cfde809114b9", "date": "Thu, 17 Oct 2024 22:41:49 GMT"} }) sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9472024-10-17T22:41:49.971ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34049 method = GET path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9482024-10-17T22:41:49.971ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34049 method = POST path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9492024-10-17T22:41:49.971ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34049 method = DELETE path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9502024-10-17T22:41:49.971ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34049 method = GET path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9512024-10-17T22:41:49.971ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34049 method = GET path = /crucible/0/regions/{id}/snapshots server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9522024-10-17T22:41:49.971ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34049 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9532024-10-17T22:41:49.971ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34049 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9542024-10-17T22:41:49.972ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34049 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9552024-10-17T22:41:49.972ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34049 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9562024-10-17T22:41:49.973ZINFOtest_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:34049 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9572024-10-17T22:41:49.973ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:34049 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9582024-10-17T22:41:49.973ZINFOtest_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:34049 kind = storage server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9592024-10-17T22:41:49.973ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35687 method = GET path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9602024-10-17T22:41:49.973ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35687 method = POST path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9612024-10-17T22:41:49.973ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35687 method = DELETE path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9622024-10-17T22:41:49.973ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35687 method = GET path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9632024-10-17T22:41:49.973ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35687 method = GET path = /crucible/0/regions/{id}/snapshots server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9642024-10-17T22:41:49.973ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35687 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9652024-10-17T22:41:49.973ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35687 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9662024-10-17T22:41:49.973ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35687 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9672024-10-17T22:41:49.973ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35687 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9682024-10-17T22:41:49.973ZINFOtest_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:35687 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9692024-10-17T22:41:49.973ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:35687 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9702024-10-17T22:41:49.974ZINFOtest_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:35687 kind = storage server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9712024-10-17T22:41:49.974ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43073 method = GET path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9722024-10-17T22:41:49.974ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43073 method = POST path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9732024-10-17T22:41:49.974ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43073 method = DELETE path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9742024-10-17T22:41:49.974ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43073 method = GET path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9752024-10-17T22:41:49.974ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43073 method = GET path = /crucible/0/regions/{id}/snapshots server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9762024-10-17T22:41:49.974ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43073 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9772024-10-17T22:41:49.974ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43073 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9782024-10-17T22:41:49.975ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43073 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9792024-10-17T22:41:49.975ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43073 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9802024-10-17T22:41:49.975ZINFOtest_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:43073 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9812024-10-17T22:41:49.975ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:43073 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9822024-10-17T22:41:49.975ZINFOtest_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:43073 kind = storage server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9832024-10-17T22:41:49.975ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40939 method = GET path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9842024-10-17T22:41:49.975ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40939 method = POST path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9852024-10-17T22:41:49.975ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40939 method = DELETE path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9862024-10-17T22:41:49.976ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40939 method = GET path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9872024-10-17T22:41:49.976ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40939 method = GET path = /crucible/0/regions/{id}/snapshots server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9882024-10-17T22:41:49.976ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40939 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9892024-10-17T22:41:49.976ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40939 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9902024-10-17T22:41:49.976ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40939 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9912024-10-17T22:41:49.976ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40939 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9922024-10-17T22:41:49.976ZINFOtest_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:40939 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9932024-10-17T22:41:49.976ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:40939 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9942024-10-17T22:41:49.976ZINFOtest_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:40939 kind = storage server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9952024-10-17T22:41:49.976ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35423 method = GET path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9962024-10-17T22:41:49.976ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35423 method = POST path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9972024-10-17T22:41:49.976ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35423 method = DELETE path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9982024-10-17T22:41:49.977ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35423 method = GET path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9992024-10-17T22:41:49.977ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35423 method = GET path = /crucible/0/regions/{id}/snapshots server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10002024-10-17T22:41:49.977ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35423 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10012024-10-17T22:41:49.978ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35423 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10022024-10-17T22:41:49.978ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35423 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10032024-10-17T22:41:49.978ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35423 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10042024-10-17T22:41:49.978ZINFOtest_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:35423 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10052024-10-17T22:41:49.978ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:35423 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10062024-10-17T22:41:49.978ZINFOtest_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:35423 kind = storage server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10072024-10-17T22:41:49.978ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34763 method = GET path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10082024-10-17T22:41:49.979ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34763 method = POST path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10092024-10-17T22:41:49.979ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34763 method = DELETE path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10102024-10-17T22:41:49.979ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34763 method = GET path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10112024-10-17T22:41:49.979ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34763 method = GET path = /crucible/0/regions/{id}/snapshots server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10122024-10-17T22:41:49.979ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34763 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10132024-10-17T22:41:49.979ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34763 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10142024-10-17T22:41:49.979ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34763 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10152024-10-17T22:41:49.979ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34763 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10162024-10-17T22:41:49.979ZINFOtest_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:34763 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10172024-10-17T22:41:49.979ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:34763 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10182024-10-17T22:41:49.979ZINFOtest_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:34763 kind = storage server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10192024-10-17T22:41:49.979ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38175 method = GET path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10202024-10-17T22:41:49.979ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38175 method = POST path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10212024-10-17T22:41:49.979ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38175 method = DELETE path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10222024-10-17T22:41:49.979ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38175 method = GET path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10232024-10-17T22:41:49.980ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38175 method = GET path = /crucible/0/regions/{id}/snapshots server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10242024-10-17T22:41:49.980ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38175 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10252024-10-17T22:41:49.980ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38175 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10262024-10-17T22:41:49.980ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38175 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10272024-10-17T22:41:49.980ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38175 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10282024-10-17T22:41:49.980ZINFOtest_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:38175 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10292024-10-17T22:41:49.980ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:38175 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10302024-10-17T22:41:49.980ZINFOtest_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:38175 kind = storage server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10312024-10-17T22:41:49.980ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46521 method = GET path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10322024-10-17T22:41:49.980ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46521 method = POST path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10332024-10-17T22:41:49.980ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46521 method = DELETE path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10342024-10-17T22:41:49.980ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46521 method = GET path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10352024-10-17T22:41:49.980ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46521 method = GET path = /crucible/0/regions/{id}/snapshots server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10362024-10-17T22:41:49.980ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46521 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10372024-10-17T22:41:49.981ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46521 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10382024-10-17T22:41:49.981ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46521 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10392024-10-17T22:41:49.981ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46521 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10402024-10-17T22:41:49.981ZINFOtest_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:46521 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10412024-10-17T22:41:49.981ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:46521 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10422024-10-17T22:41:49.981ZINFOtest_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:46521 kind = storage server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10432024-10-17T22:41:49.982ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43247 method = GET path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10442024-10-17T22:41:49.985ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43247 method = POST path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10452024-10-17T22:41:49.986ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43247 method = DELETE path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10462024-10-17T22:41:49.986ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43247 method = GET path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10472024-10-17T22:41:49.986ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43247 method = GET path = /crucible/0/regions/{id}/snapshots server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10482024-10-17T22:41:49.986ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43247 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10492024-10-17T22:41:49.987ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43247 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10502024-10-17T22:41:49.987ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43247 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10512024-10-17T22:41:49.987ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43247 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10522024-10-17T22:41:49.987ZINFOtest_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:43247 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10532024-10-17T22:41:49.987ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:43247 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10542024-10-17T22:41:49.987ZINFOtest_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:43247 kind = storage server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10552024-10-17T22:41:49.987ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:44307 method = GET path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10562024-10-17T22:41:49.987ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:44307 method = POST path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10572024-10-17T22:41:49.987ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:44307 method = DELETE path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10582024-10-17T22:41:49.987ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:44307 method = GET path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10592024-10-17T22:41:49.987ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:44307 method = GET path = /crucible/0/regions/{id}/snapshots server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10602024-10-17T22:41:49.987ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:44307 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10612024-10-17T22:41:49.987ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:44307 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10622024-10-17T22:41:49.987ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:44307 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10632024-10-17T22:41:49.987ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:44307 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10642024-10-17T22:41:49.987ZINFOtest_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:44307 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10652024-10-17T22:41:49.987ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:44307 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10662024-10-17T22:41:49.987ZINFOtest_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:44307 kind = storage server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10672024-10-17T22:41:49.989ZDEBGtest_metrics (init_with_steps): Running step start_sled2
10682024-10-17T22:41:49.989ZINFOtest_metrics (omicron_sled_agent::sim::Server): setting up sled agent server sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
10692024-10-17T22:41:50.058ZINFOtest_metrics (SledAgent): created simulated sled agent server = 039be560-54cc-49e3-88df-1a29dadbf913 sim_mode = Explicit sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
10702024-10-17T22:41:50.061ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:35099 method = POST path = /boot-disk/{boot_disk}/os/write sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
10712024-10-17T22:41:50.061ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:35099 method = GET path = /boot-disk/{boot_disk}/os/write/status sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
10722024-10-17T22:41:50.061ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:35099 method = DELETE path = /boot-disk/{boot_disk}/os/write/status/{update_id} sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
10732024-10-17T22:41:50.061ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:35099 method = GET path = /bootstore/status sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
10742024-10-17T22:41:50.061ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:35099 method = POST path = /cockroachdb sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
10752024-10-17T22:41:50.061ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:35099 method = GET path = /datasets sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
10762024-10-17T22:41:50.062ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:35099 method = PUT path = /datasets sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
10772024-10-17T22:41:50.062ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:35099 method = PUT path = /disks/{disk_id} sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
10782024-10-17T22:41:50.063ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:35099 method = POST path = /disks/{disk_id}/poke sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
10792024-10-17T22:41:50.063ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:35099 method = PUT path = /eip-gateways sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
10802024-10-17T22:41:50.063ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:35099 method = GET path = /inventory sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
10812024-10-17T22:41:50.063ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:35099 method = GET path = /network-bootstore-config sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
10822024-10-17T22:41:50.063ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:35099 method = PUT path = /network-bootstore-config sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
10832024-10-17T22:41:50.063ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:35099 method = GET path = /omicron-physical-disks sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
10842024-10-17T22:41:50.063ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:35099 method = PUT path = /omicron-physical-disks sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
10852024-10-17T22:41:50.063ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:35099 method = PUT path = /omicron-zones sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
10862024-10-17T22:41:50.063ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:35099 method = GET path = /sled-identifiers sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
10872024-10-17T22:41:50.063ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:35099 method = GET path = /sled-role sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
10882024-10-17T22:41:50.063ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:35099 method = PUT path = /sleds sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
10892024-10-17T22:41:50.063ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:35099 method = POST path = /switch-ports sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
10902024-10-17T22:41:50.064ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:35099 method = GET path = /timesync sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
10912024-10-17T22:41:50.064ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:35099 method = POST path = /update sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
10922024-10-17T22:41:50.064ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:35099 method = DELETE path = /v2p sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
10932024-10-17T22:41:50.064ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:35099 method = GET path = /v2p sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
10942024-10-17T22:41:50.065ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:35099 method = PUT path = /v2p sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
10952024-10-17T22:41:50.065ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:35099 method = DELETE path = /vmms/{propolis_id} sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
10962024-10-17T22:41:50.065ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:35099 method = PUT path = /vmms/{propolis_id} sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
10972024-10-17T22:41:50.065ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:35099 method = POST path = /vmms/{propolis_id}/disks/{disk_id}/snapshot sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
10982024-10-17T22:41:50.065ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:35099 method = DELETE path = /vmms/{propolis_id}/external-ip sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
10992024-10-17T22:41:50.065ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:35099 method = PUT path = /vmms/{propolis_id}/external-ip sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
11002024-10-17T22:41:50.065ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:35099 method = POST path = /vmms/{propolis_id}/poke sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
11012024-10-17T22:41:50.065ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:35099 method = POST path = /vmms/{propolis_id}/poke-single-step sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
11022024-10-17T22:41:50.065ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:35099 method = POST path = /vmms/{propolis_id}/sim-migration-source sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
11032024-10-17T22:41:50.065ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:35099 method = GET path = /vmms/{propolis_id}/state sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
11042024-10-17T22:41:50.065ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:35099 method = PUT path = /vmms/{propolis_id}/state sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
11052024-10-17T22:41:50.065ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:35099 method = PUT path = /vpc/{vpc_id}/firewall/rules sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
11062024-10-17T22:41:50.065ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:35099 method = GET path = /vpc-routes sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
11072024-10-17T22:41:50.065ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:35099 method = PUT path = /vpc-routes sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
11082024-10-17T22:41:50.065ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:35099 method = GET path = /zones sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
11092024-10-17T22:41:50.065ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:35099 method = POST path = /zones/bundle-cleanup sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
11102024-10-17T22:41:50.065ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:35099 method = GET path = /zones/bundle-cleanup/context sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
11112024-10-17T22:41:50.065ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:35099 method = PUT path = /zones/bundle-cleanup/context sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
11122024-10-17T22:41:50.065ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:35099 method = GET path = /zones/bundle-cleanup/utilization sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
11132024-10-17T22:41:50.066ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:35099 method = GET path = /zones/bundles sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
11142024-10-17T22:41:50.066ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:35099 method = GET path = /zones/bundles/{zone_name} sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
11152024-10-17T22:41:50.066ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:35099 method = POST path = /zones/bundles/{zone_name} sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
11162024-10-17T22:41:50.066ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:35099 method = DELETE path = /zones/bundles/{zone_name}/{bundle_id} sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
11172024-10-17T22:41:50.066ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:35099 method = GET path = /zones/bundles/{zone_name}/{bundle_id} sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
11182024-10-17T22:41:50.066ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:35099 method = GET path = /zpools sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
11192024-10-17T22:41:50.066ZINFOtest_metrics (dropshot): listening local_addr = [::1]:35099 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
11202024-10-17T22:41:50.066ZDEBGtest_metrics (dropshot): successfully registered DTrace USDT probes local_addr = [::1]:35099 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
11212024-10-17T22:41:50.066ZDEBGtest_metrics (omicron_sled_agent::sim::Server): contacting server nexus sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
11222024-10-17T22:41:50.066ZDEBGtest_metrics (NexusClient): client request body = Some(Body) method = POST sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = http://127.0.0.1:33311/sled-agents/039be560-54cc-49e3-88df-1a29dadbf913
11232024-10-17T22:41:50.069ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
11242024-10-17T22:41:50.069ZDEBGtest_metrics (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
11252024-10-17T22:41:50.070ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
11262024-10-17T22:41:50.071ZWARNtest_metrics (BackgroundTasks): failed to read DNS config background_task = dns_config_internal dns_group = internal error = Internal Error: expected exactly one latest version for DNS group Internal, found 0
11272024-10-17T22:41:50.071ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 803.2689ms iteration = 1 status = {"error":"failed to read DNS config: Internal Error: expected exactly one latest version for DNS group Internal, found 0"}
11282024-10-17T22:41:50.071ZWARNtest_metrics (BackgroundTasks): failed to read Silo/DNS/TLS configuration background_task = external_endpoints error = Internal Error: expected at least one external DNS zone
11292024-10-17T22:41:50.071ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 795.338958ms iteration = 1 status = {"error":"failed to read Silo/DNS/TLS configuration: Internal Error: expected at least one external DNS zone"}
11302024-10-17T22:41:50.075ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11312024-10-17T22:41:50.078ZDEBGtest_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
11322024-10-17T22:41:50.078ZTRACtest_metrics (BackgroundTasks): successful lookup of siloed resource "Vpc" using built-in user background_task = service_firewall_rule_propagation
11332024-10-17T22:41:50.078ZTRACtest_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
11342024-10-17T22:41:50.078ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
11352024-10-17T22:41:50.078ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
11362024-10-17T22:41:50.079ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
11372024-10-17T22:41:50.079ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
11382024-10-17T22:41:50.080ZTRACtest_metrics (BackgroundTasks): no more instances in need of reincarnation background_task = instance_reincarnation reincarnation_reason = start saga failed total_found = 0
11392024-10-17T22:41:50.080ZWARNtest_metrics (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
11402024-10-17T22:41:50.080ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 787.364467ms iteration = 1 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
11412024-10-17T22:41:50.089ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11422024-10-17T22:41:50.108ZDEBGtest_metrics (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
11432024-10-17T22:41:50.108ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
11442024-10-17T22:41:50.108ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
11452024-10-17T22:41:50.109ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
11462024-10-17T22:41:50.159ZINFOtest_metrics (BackgroundTasks): VPC route manager sled b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager
11472024-10-17T22:41:50.160ZDEBGtest_metrics (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager body = None method = GET uri = http://[::1]:40953/vpc-routes
11482024-10-17T22:41:50.160ZINFOtest_metrics (dropshot_internal): accepted connection local_addr = 127.0.0.1:33311 remote_addr = 127.0.0.1:36382
11492024-10-17T22:41:50.161ZINFOtest_metrics (dropshot): accepted connection local_addr = [::1]:40953 remote_addr = [::1]:57522 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
11502024-10-17T22:41:50.161ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
11512024-10-17T22:41:50.162ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
11522024-10-17T22:41:50.162ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
11532024-10-17T22:41:50.163ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
11542024-10-17T22:41:50.164ZDEBGtest_metrics (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = []
11552024-10-17T22:41:50.164ZTRACtest_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
11562024-10-17T22:41:50.165ZTRACtest_metrics (dropshot_internal): incoming request local_addr = 127.0.0.1:33311 method = POST remote_addr = 127.0.0.1:36382 req_id = d4dee04f-51a0-482b-aa11-6322dd536b18 uri = /sled-agents/039be560-54cc-49e3-88df-1a29dadbf913
11572024-10-17T22:41:50.165ZTRACtest_metrics (dropshot): incoming request local_addr = [::1]:40953 method = GET remote_addr = [::1]:57522 req_id = e780c431-06f1-43f6-84a7-b2d69a0c66b4 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc-routes
11582024-10-17T22:41:50.166ZINFOtest_metrics (nexus): registered sled agent sled_uuid = 039be560-54cc-49e3-88df-1a29dadbf913
11592024-10-17T22:41:50.172ZINFOtest_metrics (dropshot): request completed latency_us = 6393 local_addr = [::1]:40953 method = GET remote_addr = [::1]:57522 req_id = e780c431-06f1-43f6-84a7-b2d69a0c66b4 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc-routes
11602024-10-17T22:41:50.185ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
11612024-10-17T22:41:50.185ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
11622024-10-17T22:41:50.185ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
11632024-10-17T22:41:50.187ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
11642024-10-17T22:41:50.188ZTRACtest_metrics (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: ea a9 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:64277 req_id = 74fe0393-8a46-4a44-817f-220e9ed8c5e0
11652024-10-17T22:41:50.188ZDEBGtest_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 60073,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64277 req_id = 74fe0393-8a46-4a44-817f-220e9ed8c5e0
11662024-10-17T22:41:50.189ZERROtest_metrics (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 60073, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_mgs._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 4, 95, 109, 103, 115, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_mgs._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:64277 req_id = 74fe0393-8a46-4a44-817f-220e9ed8c5e0
11672024-10-17T22:41:50.189ZDEBGtest_metrics (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager result = Ok(Response { url: "http://[::1]:40953/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "e780c431-06f1-43f6-84a7-b2d69a0c66b4", "content-length": "2", "date": "Thu, 17 Oct 2024 22:41:49 GMT"} })
11682024-10-17T22:41:50.189ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
11692024-10-17T22:41:50.189ZDEBGtest_metrics (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11702024-10-17T22:41:50.189ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11712024-10-17T22:41:50.191ZWARNtest_metrics (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: looking up MGS addresses: no record found for Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
11722024-10-17T22:41:50.191ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 230.018992ms iteration = 2 status = {"error":"failed to collect inventory: looking up MGS addresses: no record found for Query { name: Name(\\"_mgs._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }"}
11732024-10-17T22:41:50.230ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11742024-10-17T22:41:50.253ZDEBGtest_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
11752024-10-17T22:41:50.254ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
11762024-10-17T22:41:50.254ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
11772024-10-17T22:41:50.255ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
11782024-10-17T22:41:50.266ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
11792024-10-17T22:41:50.279ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
11802024-10-17T22:41:50.280ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
11812024-10-17T22:41:50.292ZTRACtest_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet
11822024-10-17T22:41:50.293ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = v2p_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
11832024-10-17T22:41:50.293ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
11842024-10-17T22:41:50.293ZDEBGtest_metrics (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
11852024-10-17T22:41:50.294ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
11862024-10-17T22:41:50.295ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
11872024-10-17T22:41:50.297ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
11882024-10-17T22:41:50.297ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
11892024-10-17T22:41:50.297ZDEBGtest_metrics (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11902024-10-17T22:41:50.302ZDEBGtest_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
11912024-10-17T22:41:50.302ZTRACtest_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet
11922024-10-17T22:41:50.302ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = v2p_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
11932024-10-17T22:41:50.302ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
11942024-10-17T22:41:50.302ZDEBGtest_metrics (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
11952024-10-17T22:41:50.303ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
11962024-10-17T22:41:50.308ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
11972024-10-17T22:41:50.308ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
11982024-10-17T22:41:50.309ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
11992024-10-17T22:41:50.309ZDEBGtest_metrics (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12002024-10-17T22:41:50.314ZDEBGtest_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
12012024-10-17T22:41:50.314ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
12022024-10-17T22:41:50.314ZDEBGtest_metrics (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
12032024-10-17T22:41:50.315ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
12042024-10-17T22:41:50.325ZTRACtest_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
12052024-10-17T22:41:50.325ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
12062024-10-17T22:41:50.325ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
12072024-10-17T22:41:50.325ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12082024-10-17T22:41:50.326ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12092024-10-17T22:41:50.328ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
12102024-10-17T22:41:50.328ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
12112024-10-17T22:41:50.329ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12122024-10-17T22:41:50.329ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12132024-10-17T22:41:50.330ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
12142024-10-17T22:41:50.330ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
12152024-10-17T22:41:50.330ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12162024-10-17T22:41:50.331ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12172024-10-17T22:41:50.333ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12182024-10-17T22:41:50.343ZDEBGtest_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
12192024-10-17T22:41:50.343ZTRACtest_metrics (BackgroundTasks): successful lookup of siloed resource "Vpc" using built-in user background_task = service_firewall_rule_propagation
12202024-10-17T22:41:50.343ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
12212024-10-17T22:41:50.343ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12222024-10-17T22:41:50.344ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12232024-10-17T22:41:50.344ZDEBGtest_metrics (BackgroundTasks): activating background_task = inventory_collection iteration = 3 reason = Signaled
12242024-10-17T22:41:50.345ZTRACtest_metrics (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
12252024-10-17T22:41:50.345ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
12262024-10-17T22:41:50.345ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
12272024-10-17T22:41:50.345ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
12282024-10-17T22:41:50.345ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
12292024-10-17T22:41:50.345ZINFOtest_metrics (dropshot_internal): request completed latency_us = 180158 local_addr = 127.0.0.1:33311 method = POST remote_addr = 127.0.0.1:36382 req_id = d4dee04f-51a0-482b-aa11-6322dd536b18 response_code = 204 uri = /sled-agents/039be560-54cc-49e3-88df-1a29dadbf913
12302024-10-17T22:41:50.346ZDEBGtest_metrics (NexusClient): client response result = Ok(Response { url: "http://127.0.0.1:33311/sled-agents/039be560-54cc-49e3-88df-1a29dadbf913", status: 204, headers: {"x-request-id": "d4dee04f-51a0-482b-aa11-6322dd536b18", "date": "Thu, 17 Oct 2024 22:41:50 GMT"} }) sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12312024-10-17T22:41:50.348ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:42209 method = GET path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12322024-10-17T22:41:50.349ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:42209 method = POST path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12332024-10-17T22:41:50.349ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:42209 method = DELETE path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12342024-10-17T22:41:50.349ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:42209 method = GET path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12352024-10-17T22:41:50.349ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:42209 method = GET path = /crucible/0/regions/{id}/snapshots server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12362024-10-17T22:41:50.350ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:42209 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12372024-10-17T22:41:50.350ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:42209 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12382024-10-17T22:41:50.350ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:42209 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12392024-10-17T22:41:50.350ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:42209 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12402024-10-17T22:41:50.350ZINFOtest_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:42209 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12412024-10-17T22:41:50.350ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:42209 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12422024-10-17T22:41:50.350ZINFOtest_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:42209 kind = storage server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12432024-10-17T22:41:50.350ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46419 method = GET path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12442024-10-17T22:41:50.351ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46419 method = POST path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12452024-10-17T22:41:50.351ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46419 method = DELETE path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12462024-10-17T22:41:50.351ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46419 method = GET path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12472024-10-17T22:41:50.351ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46419 method = GET path = /crucible/0/regions/{id}/snapshots server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12482024-10-17T22:41:50.351ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46419 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12492024-10-17T22:41:50.351ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46419 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12502024-10-17T22:41:50.351ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46419 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12512024-10-17T22:41:50.352ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46419 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12522024-10-17T22:41:50.352ZINFOtest_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:46419 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12532024-10-17T22:41:50.352ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:46419 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12542024-10-17T22:41:50.352ZINFOtest_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:46419 kind = storage server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12552024-10-17T22:41:50.352ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39967 method = GET path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12562024-10-17T22:41:50.352ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39967 method = POST path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12572024-10-17T22:41:50.352ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39967 method = DELETE path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12582024-10-17T22:41:50.353ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39967 method = GET path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12592024-10-17T22:41:50.353ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39967 method = GET path = /crucible/0/regions/{id}/snapshots server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12602024-10-17T22:41:50.353ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39967 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12612024-10-17T22:41:50.353ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39967 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12622024-10-17T22:41:50.353ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39967 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12632024-10-17T22:41:50.354ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39967 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12642024-10-17T22:41:50.354ZINFOtest_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:39967 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12652024-10-17T22:41:50.355ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:39967 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12662024-10-17T22:41:50.355ZINFOtest_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:39967 kind = storage server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12672024-10-17T22:41:50.355ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43863 method = GET path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12682024-10-17T22:41:50.355ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43863 method = POST path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12692024-10-17T22:41:50.355ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43863 method = DELETE path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12702024-10-17T22:41:50.355ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43863 method = GET path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12712024-10-17T22:41:50.355ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43863 method = GET path = /crucible/0/regions/{id}/snapshots server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12722024-10-17T22:41:50.355ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43863 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12732024-10-17T22:41:50.355ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43863 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12742024-10-17T22:41:50.355ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43863 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12752024-10-17T22:41:50.355ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43863 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12762024-10-17T22:41:50.355ZINFOtest_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:43863 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12772024-10-17T22:41:50.355ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:43863 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12782024-10-17T22:41:50.355ZINFOtest_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:43863 kind = storage server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12792024-10-17T22:41:50.356ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35933 method = GET path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12802024-10-17T22:41:50.356ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35933 method = POST path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12812024-10-17T22:41:50.356ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35933 method = DELETE path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12822024-10-17T22:41:50.356ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35933 method = GET path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12832024-10-17T22:41:50.356ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35933 method = GET path = /crucible/0/regions/{id}/snapshots server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12842024-10-17T22:41:50.356ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35933 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12852024-10-17T22:41:50.356ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35933 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12862024-10-17T22:41:50.356ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35933 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12872024-10-17T22:41:50.356ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35933 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12882024-10-17T22:41:50.356ZINFOtest_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:35933 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12892024-10-17T22:41:50.357ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:35933 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12902024-10-17T22:41:50.357ZINFOtest_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:35933 kind = storage server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12912024-10-17T22:41:50.357ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34137 method = GET path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12922024-10-17T22:41:50.357ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34137 method = POST path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12932024-10-17T22:41:50.357ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34137 method = DELETE path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12942024-10-17T22:41:50.357ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34137 method = GET path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12952024-10-17T22:41:50.357ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34137 method = GET path = /crucible/0/regions/{id}/snapshots server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12962024-10-17T22:41:50.357ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34137 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12972024-10-17T22:41:50.358ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34137 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12982024-10-17T22:41:50.358ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34137 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12992024-10-17T22:41:50.358ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34137 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13002024-10-17T22:41:50.358ZINFOtest_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:34137 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13012024-10-17T22:41:50.358ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:34137 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13022024-10-17T22:41:50.358ZINFOtest_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:34137 kind = storage server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13032024-10-17T22:41:50.358ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:33599 method = GET path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13042024-10-17T22:41:50.358ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:33599 method = POST path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13052024-10-17T22:41:50.358ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:33599 method = DELETE path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13062024-10-17T22:41:50.358ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:33599 method = GET path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13072024-10-17T22:41:50.358ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:33599 method = GET path = /crucible/0/regions/{id}/snapshots server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13082024-10-17T22:41:50.358ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:33599 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13092024-10-17T22:41:50.358ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:33599 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13102024-10-17T22:41:50.358ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:33599 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13112024-10-17T22:41:50.358ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:33599 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13122024-10-17T22:41:50.358ZINFOtest_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:33599 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13132024-10-17T22:41:50.358ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:33599 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13142024-10-17T22:41:50.359ZINFOtest_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:33599 kind = storage server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13152024-10-17T22:41:50.359ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45723 method = GET path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13162024-10-17T22:41:50.359ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45723 method = POST path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13172024-10-17T22:41:50.373ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45723 method = DELETE path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13182024-10-17T22:41:50.373ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45723 method = GET path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13192024-10-17T22:41:50.373ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45723 method = GET path = /crucible/0/regions/{id}/snapshots server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13202024-10-17T22:41:50.373ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45723 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13212024-10-17T22:41:50.373ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45723 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13222024-10-17T22:41:50.373ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45723 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13232024-10-17T22:41:50.373ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45723 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13242024-10-17T22:41:50.373ZINFOtest_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:45723 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13252024-10-17T22:41:50.373ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:45723 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13262024-10-17T22:41:50.373ZINFOtest_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:45723 kind = storage server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13272024-10-17T22:41:50.373ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38373 method = GET path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13282024-10-17T22:41:50.373ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38373 method = POST path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13292024-10-17T22:41:50.373ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38373 method = DELETE path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13302024-10-17T22:41:50.373ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38373 method = GET path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13312024-10-17T22:41:50.374ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38373 method = GET path = /crucible/0/regions/{id}/snapshots server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13322024-10-17T22:41:50.374ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38373 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13332024-10-17T22:41:50.374ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38373 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13342024-10-17T22:41:50.374ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38373 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13352024-10-17T22:41:50.374ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38373 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13362024-10-17T22:41:50.374ZINFOtest_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:38373 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13372024-10-17T22:41:50.374ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:38373 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13382024-10-17T22:41:50.374ZINFOtest_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:38373 kind = storage server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13392024-10-17T22:41:50.374ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40641 method = GET path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13402024-10-17T22:41:50.374ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40641 method = POST path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13412024-10-17T22:41:50.375ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40641 method = DELETE path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13422024-10-17T22:41:50.375ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40641 method = GET path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13432024-10-17T22:41:50.375ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40641 method = GET path = /crucible/0/regions/{id}/snapshots server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13442024-10-17T22:41:50.376ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40641 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13452024-10-17T22:41:50.376ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40641 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13462024-10-17T22:41:50.376ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40641 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13472024-10-17T22:41:50.376ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40641 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13482024-10-17T22:41:50.376ZINFOtest_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:40641 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13492024-10-17T22:41:50.376ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:40641 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13502024-10-17T22:41:50.376ZINFOtest_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:40641 kind = storage server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13512024-10-17T22:41:50.377ZDEBGtest_metrics (init_with_steps): Running step start_crucible_pantry
13522024-10-17T22:41:50.377ZDEBGtest_metrics (dropshot): registered endpoint kind = pantry local_addr = [::1]:38587 method = GET path = /crucible/pantry/0 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
13532024-10-17T22:41:50.377ZDEBGtest_metrics (dropshot): registered endpoint kind = pantry local_addr = [::1]:38587 method = GET path = /crucible/pantry/0/job/{id}/is-finished sled_id = b6d65341-167c-41df-9b5c-41cded99c229
13542024-10-17T22:41:50.377ZDEBGtest_metrics (dropshot): registered endpoint kind = pantry local_addr = [::1]:38587 method = GET path = /crucible/pantry/0/job/{id}/ok sled_id = b6d65341-167c-41df-9b5c-41cded99c229
13552024-10-17T22:41:50.377ZDEBGtest_metrics (dropshot): registered endpoint kind = pantry local_addr = [::1]:38587 method = DELETE path = /crucible/pantry/0/volume/{id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229
13562024-10-17T22:41:50.377ZDEBGtest_metrics (dropshot): registered endpoint kind = pantry local_addr = [::1]:38587 method = GET path = /crucible/pantry/0/volume/{id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229
13572024-10-17T22:41:50.377ZDEBGtest_metrics (dropshot): registered endpoint kind = pantry local_addr = [::1]:38587 method = POST path = /crucible/pantry/0/volume/{id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229
13582024-10-17T22:41:50.377ZDEBGtest_metrics (dropshot): registered endpoint kind = pantry local_addr = [::1]:38587 method = POST path = /crucible/pantry/0/volume/{id}/background sled_id = b6d65341-167c-41df-9b5c-41cded99c229
13592024-10-17T22:41:50.377ZDEBGtest_metrics (dropshot): registered endpoint kind = pantry local_addr = [::1]:38587 method = POST path = /crucible/pantry/0/volume/{id}/bulk-write sled_id = b6d65341-167c-41df-9b5c-41cded99c229
13602024-10-17T22:41:50.377ZDEBGtest_metrics (dropshot): registered endpoint kind = pantry local_addr = [::1]:38587 method = POST path = /crucible/pantry/0/volume/{id}/import-from-url sled_id = b6d65341-167c-41df-9b5c-41cded99c229
13612024-10-17T22:41:50.377ZDEBGtest_metrics (dropshot): registered endpoint kind = pantry local_addr = [::1]:38587 method = POST path = /crucible/pantry/0/volume/{id}/scrub sled_id = b6d65341-167c-41df-9b5c-41cded99c229
13622024-10-17T22:41:50.377ZDEBGtest_metrics (dropshot): registered endpoint kind = pantry local_addr = [::1]:38587 method = POST path = /crucible/pantry/0/volume/{id}/snapshot sled_id = b6d65341-167c-41df-9b5c-41cded99c229
13632024-10-17T22:41:50.378ZINFOtest_metrics (dropshot): listening kind = pantry local_addr = [::1]:38587 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
13642024-10-17T22:41:50.378ZDEBGtest_metrics (dropshot): successfully registered DTrace USDT probes kind = pantry local_addr = [::1]:38587 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
13652024-10-17T22:41:50.378ZINFOtest_metrics (omicron_sled_agent::sim::Server): Started Simulated Crucible Pantry address = [::1]:38587 kind = pantry sled_id = b6d65341-167c-41df-9b5c-41cded99c229
13662024-10-17T22:41:50.378ZDEBGtest_metrics (init_with_steps): Running step populate_internal_dns
13672024-10-17T22:41:50.378ZDEBGtest_metrics: Populating Internal DNS
13682024-10-17T22:41:50.443ZINFOtest_metrics: DNS population: DnsConfigParams { generation: Generation( 1, ), time_created: 2024-10-17T22:41:50.440549881Z, zones: [ DnsConfigZone { zone_name: "control-plane.oxide.internal", records: { "_clickhouse-native._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 9000, target: "49fc5e60-4087-447e-95ed-2a48b4c7e6bc.host.control-plane.oxide.internal", }, ), ], "_crucible-pantry._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 38587, target: "4cc306e2-a646-4794-a23e-b059c98afee3.host.control-plane.oxide.internal", }, ), ], "_external-dns._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 36177, target: "0e0c52d8-7b87-4b83-9828-05f7d187ffb1.host.control-plane.oxide.internal", }, ), ], "996eea1f-0c18-450c-8228-39db7dc3685a.host": [ Aaaa( ::1, ), ], "_mgd._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 37593, target: "dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal", }, ), Srv( Srv { prio: 0, weight: 0, port: 39577, target: "dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal", }, ), ], "_mgs._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 33937, target: "dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal", }, ), Srv( Srv { prio: 0, weight: 0, port: 44607, target: "dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal", }, ), ], "4cc306e2-a646-4794-a23e-b059c98afee3.host": [ Aaaa( ::1, ), ], "017ff6a2-3b72-4882-9b0b-97601068f99c.host": [ Aaaa( ::ffff:127.0.0.1, ), ], "_dendrite._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 36237, target: "dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal", }, ), Srv( Srv { prio: 0, weight: 0, port: 36313, target: "dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal", }, ), ], "dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host": [ Aaaa( ::1, ), ], "49fc5e60-4087-447e-95ed-2a48b4c7e6bc.host": [ Aaaa( ::1, ), ], "_clickhouse._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 35287, target: "49fc5e60-4087-447e-95ed-2a48b4c7e6bc.host.control-plane.oxide.internal", }, ), ], "0e0c52d8-7b87-4b83-9828-05f7d187ffb1.host": [ Aaaa( ::1, ), ], "_cockroach._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 41953, target: "060ec4fd-2910-4e74-b026-e672be295c2e.host.control-plane.oxide.internal", }, ), ], "_nameservice._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 41599, target: "996eea1f-0c18-450c-8228-39db7dc3685a.host.control-plane.oxide.internal", }, ), ], "060ec4fd-2910-4e74-b026-e672be295c2e.host": [ Aaaa( ::1, ), ], "dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host": [ Aaaa( ::1, ), ], "_nexus._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 33311, target: "017ff6a2-3b72-4882-9b0b-97601068f99c.host.control-plane.oxide.internal", }, ), ], }, }, ], }
13692024-10-17T22:41:50.443ZDEBGtest_metrics: client request body = Some(Body) method = PUT uri = http://[::1]:41599/config
13702024-10-17T22:41:50.444ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13712024-10-17T22:41:50.444ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13722024-10-17T22:41:50.444ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13732024-10-17T22:41:50.447ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13742024-10-17T22:41:50.458ZDEBGtest_metrics (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
13752024-10-17T22:41:50.458ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
13762024-10-17T22:41:50.458ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
13772024-10-17T22:41:50.459ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
13782024-10-17T22:41:50.461ZINFOtest_metrics (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager sled = sim-b6d65341
13792024-10-17T22:41:50.461ZDEBGtest_metrics (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[::1]:40953/eip-gateways
13802024-10-17T22:41:50.461ZINFOtest_metrics (http): accepted connection kind = dns local_addr = [::1]:41599 remote_addr = [::1]:52938
13812024-10-17T22:41:50.461ZTRACtest_metrics (dropshot): incoming request local_addr = [::1]:40953 method = PUT remote_addr = [::1]:57522 req_id = 72982225-c01f-4077-a21c-8de5df0f63f7 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /eip-gateways
13822024-10-17T22:41:50.464ZINFOtest_metrics (dropshot): request completed latency_us = 2168 local_addr = [::1]:40953 method = PUT remote_addr = [::1]:57522 req_id = 72982225-c01f-4077-a21c-8de5df0f63f7 response_code = 204 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /eip-gateways
13832024-10-17T22:41:50.465ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13842024-10-17T22:41:50.465ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13852024-10-17T22:41:50.465ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13862024-10-17T22:41:50.574ZDEBGtest_metrics (BackgroundTasks): client request SledAgent = 039be560-54cc-49e3-88df-1a29dadbf913 background_task = v2p_manager body = None method = GET uri = http://[::1]:35099/v2p
13872024-10-17T22:41:50.575ZDEBGtest_metrics (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = []
13882024-10-17T22:41:50.575ZTRACtest_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
13892024-10-17T22:41:50.576ZINFOtest_metrics (dropshot): accepted connection local_addr = [::1]:35099 remote_addr = [::1]:59828 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13902024-10-17T22:41:50.576ZDEBGtest_metrics (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager result = Ok(Response { url: "http://[::1]:40953/eip-gateways", status: 204, headers: {"x-request-id": "72982225-c01f-4077-a21c-8de5df0f63f7", "date": "Thu, 17 Oct 2024 22:41:50 GMT"} })
13912024-10-17T22:41:50.577ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 1.282843904s iteration = 1 status = {}
13922024-10-17T22:41:50.580ZTRACtest_metrics (dropshot): incoming request local_addr = [::1]:35099 method = GET remote_addr = [::1]:59828 req_id = a7fb0f06-e44a-4d55-9391-6a89c1e74a74 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /v2p
13932024-10-17T22:41:50.580ZTRACtest_metrics (http): incoming request kind = dns local_addr = [::1]:41599 method = PUT remote_addr = [::1]:52938 req_id = ed6383b5-7b11-461c-986f-897cfc059c3f uri = /config
13942024-10-17T22:41:50.582ZTRACtest_metrics (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: a2 5d 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .]..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:59363 req_id = 95a2362b-1fd6-4aa5-9757-8e86d13ad805
13952024-10-17T22:41:50.582ZDEBGtest_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 41565,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:59363 req_id = 95a2362b-1fd6-4aa5-9757-8e86d13ad805
13962024-10-17T22:41:50.582ZERROtest_metrics (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 41565, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_mgs._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 4, 95, 109, 103, 115, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_mgs._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:59363 req_id = 95a2362b-1fd6-4aa5-9757-8e86d13ad805
13972024-10-17T22:41:50.582ZINFOtest_metrics (store): attempting generation update new_generation = 1 req_id = ed6383b5-7b11-461c-986f-897cfc059c3f
13982024-10-17T22:41:50.582ZINFOtest_metrics (store): pruning trees for generations newer than 0
13992024-10-17T22:41:50.584ZDEBGtest_metrics (store): creating tree tree_name = generation_1_zone_control-plane.oxide.internal
14002024-10-17T22:41:50.584ZINFOtest_metrics (dropshot): request completed latency_us = 2157 local_addr = [::1]:35099 method = GET remote_addr = [::1]:59828 req_id = a7fb0f06-e44a-4d55-9391-6a89c1e74a74 response_code = 200 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /v2p
14012024-10-17T22:41:50.584ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14022024-10-17T22:41:50.584ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14032024-10-17T22:41:50.584ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14042024-10-17T22:41:50.584ZWARNtest_metrics (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: looking up MGS addresses: no record found for Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
14052024-10-17T22:41:50.585ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 237.781209ms iteration = 3 status = {"error":"failed to collect inventory: looking up MGS addresses: no record found for Query { name: Name(\\"_mgs._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }"}
14062024-10-17T22:41:50.585ZDEBGtest_metrics (store): updating current config
14072024-10-17T22:41:50.585ZDEBGtest_metrics (store): flushing default tree
14082024-10-17T22:41:50.585ZDEBGtest_metrics (BackgroundTasks): client response SledAgent = 039be560-54cc-49e3-88df-1a29dadbf913 background_task = v2p_manager result = Ok(Response { url: "http://[::1]:35099/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "a7fb0f06-e44a-4d55-9391-6a89c1e74a74", "content-length": "2", "date": "Thu, 17 Oct 2024 22:41:50 GMT"} })
14092024-10-17T22:41:50.585ZINFOtest_metrics (BackgroundTasks): found opte v2p mappings background_task = v2p_manager interfaces = {} sled = sim-039be560
14102024-10-17T22:41:50.585ZINFOtest_metrics (BackgroundTasks): v2p mappings to delete background_task = v2p_manager mappings = [] sled = sim-039be560
14112024-10-17T22:41:50.587ZINFOtest_metrics (BackgroundTasks): v2p mappings to add background_task = v2p_manager mappings = [] sled = sim-039be560
14122024-10-17T22:41:50.587ZDEBGtest_metrics (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = v2p_manager body = None method = GET uri = http://[::1]:40953/v2p
14132024-10-17T22:41:50.588ZTRACtest_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") }
14142024-10-17T22:41:50.588ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
14152024-10-17T22:41:50.590ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14162024-10-17T22:41:50.590ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14172024-10-17T22:41:50.590ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14182024-10-17T22:41:50.590ZINFOtest_metrics (store): pruning trees for generations older than 1 keep = 3
14192024-10-17T22:41:50.590ZINFOtest_metrics (store): updated generation new_generation = 1 req_id = ed6383b5-7b11-461c-986f-897cfc059c3f
14202024-10-17T22:41:50.590ZINFOtest_metrics (http): request completed kind = dns latency_us = 11208 local_addr = [::1]:41599 method = PUT remote_addr = [::1]:52938 req_id = ed6383b5-7b11-461c-986f-897cfc059c3f response_code = 204 uri = /config
14212024-10-17T22:41:50.590ZINFOtest_metrics (dropshot): accepted connection local_addr = [::1]:40953 remote_addr = [::1]:57524 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
14222024-10-17T22:41:50.590ZDEBGtest_metrics: client response result = Ok(Response { url: "http://[::1]:41599/config", status: 204, headers: {"x-request-id": "ed6383b5-7b11-461c-986f-897cfc059c3f", "date": "Thu, 17 Oct 2024 22:41:50 GMT"} })
14232024-10-17T22:41:50.595ZDEBGtest_metrics (init_with_steps): Running step configure_sled_agent1
14242024-10-17T22:41:50.630ZDEBGtest_metrics: client request body = Some(Body) method = PUT uri = http://[::1]:40953/omicron-zones
14252024-10-17T22:41:50.631ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
14262024-10-17T22:41:50.631ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14272024-10-17T22:41:50.632ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14282024-10-17T22:41:50.632ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14292024-10-17T22:41:50.632ZTRACtest_metrics (dropshot): incoming request local_addr = [::1]:40953 method = GET remote_addr = [::1]:57524 req_id = a3751602-b705-4e36-8473-364edd039f35 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /v2p
14302024-10-17T22:41:50.632ZINFOtest_metrics (dropshot): request completed latency_us = 667 local_addr = [::1]:40953 method = GET remote_addr = [::1]:57524 req_id = a3751602-b705-4e36-8473-364edd039f35 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /v2p
14312024-10-17T22:41:50.632ZINFOtest_metrics (dropshot): accepted connection local_addr = [::1]:40953 remote_addr = [::1]:57526 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
14322024-10-17T22:41:50.633ZDEBGtest_metrics (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = v2p_manager result = Ok(Response { url: "http://[::1]:40953/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "a3751602-b705-4e36-8473-364edd039f35", "content-length": "2", "date": "Thu, 17 Oct 2024 22:41:50 GMT"} })
14332024-10-17T22:41:50.634ZINFOtest_metrics (BackgroundTasks): found opte v2p mappings background_task = v2p_manager interfaces = {} sled = sim-b6d65341
14342024-10-17T22:41:50.634ZINFOtest_metrics (BackgroundTasks): v2p mappings to delete background_task = v2p_manager mappings = [] sled = sim-b6d65341
14352024-10-17T22:41:50.634ZINFOtest_metrics (BackgroundTasks): v2p mappings to add background_task = v2p_manager mappings = [] sled = sim-b6d65341
14362024-10-17T22:41:50.634ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 1.347105339s iteration = 1 status = {}
14372024-10-17T22:41:50.636ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
14382024-10-17T22:41:50.636ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14392024-10-17T22:41:50.636ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14402024-10-17T22:41:50.637ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14412024-10-17T22:41:50.637ZTRACtest_metrics (dropshot): incoming request local_addr = [::1]:40953 method = PUT remote_addr = [::1]:57526 req_id = e87396fb-92ac-4de3-b62c-e2dd610d42f6 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /omicron-zones
14422024-10-17T22:41:50.639ZINFOtest_metrics (dropshot): request completed latency_us = 1455 local_addr = [::1]:40953 method = PUT remote_addr = [::1]:57526 req_id = e87396fb-92ac-4de3-b62c-e2dd610d42f6 response_code = 204 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /omicron-zones
14432024-10-17T22:41:50.642ZDEBGtest_metrics: client response result = Ok(Response { url: "http://[::1]:40953/omicron-zones", status: 204, headers: {"x-request-id": "e87396fb-92ac-4de3-b62c-e2dd610d42f6", "date": "Thu, 17 Oct 2024 22:41:50 GMT"} })
14442024-10-17T22:41:50.643ZDEBGtest_metrics: client request body = Some(Body) method = PUT uri = http://[::1]:40953/network-bootstore-config
14452024-10-17T22:41:50.643ZTRACtest_metrics (dropshot): incoming request local_addr = [::1]:40953 method = PUT remote_addr = [::1]:57526 req_id = b62fffa3-3ba0-471b-856c-647ccc35f6f5 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /network-bootstore-config
14462024-10-17T22:41:50.643ZINFOtest_metrics (dropshot): request completed latency_us = 267 local_addr = [::1]:40953 method = PUT remote_addr = [::1]:57526 req_id = b62fffa3-3ba0-471b-856c-647ccc35f6f5 response_code = 204 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /network-bootstore-config
14472024-10-17T22:41:50.643ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14482024-10-17T22:41:50.648ZDEBGtest_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
14492024-10-17T22:41:50.648ZTRACtest_metrics (BackgroundTasks): successful lookup of siloed resource "VpcSubnet" using built-in user background_task = service_firewall_rule_propagation
14502024-10-17T22:41:50.648ZTRACtest_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") }
14512024-10-17T22:41:50.648ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
14522024-10-17T22:41:50.648ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14532024-10-17T22:41:50.648ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14542024-10-17T22:41:50.649ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14552024-10-17T22:41:50.649ZDEBGtest_metrics: client response result = Ok(Response { url: "http://[::1]:40953/network-bootstore-config", status: 204, headers: {"x-request-id": "b62fffa3-3ba0-471b-856c-647ccc35f6f5", "date": "Thu, 17 Oct 2024 22:41:50 GMT"} })
14562024-10-17T22:41:50.651ZDEBGtest_metrics (init_with_steps): Running step configure_sled_agent2
14572024-10-17T22:41:50.689ZDEBGtest_metrics: client request body = Some(Body) method = PUT uri = http://[::1]:35099/omicron-zones
14582024-10-17T22:41:50.690ZINFOtest_metrics (dropshot): accepted connection local_addr = [::1]:35099 remote_addr = [::1]:59830 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
14592024-10-17T22:41:50.691ZTRACtest_metrics (dropshot): incoming request local_addr = [::1]:35099 method = PUT remote_addr = [::1]:59830 req_id = 4a7f80dc-e5f5-402b-9caa-5e32e8067ac3 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /omicron-zones
14602024-10-17T22:41:50.691ZINFOtest_metrics (dropshot): request completed latency_us = 324 local_addr = [::1]:35099 method = PUT remote_addr = [::1]:59830 req_id = 4a7f80dc-e5f5-402b-9caa-5e32e8067ac3 response_code = 204 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /omicron-zones
14612024-10-17T22:41:50.693ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
14622024-10-17T22:41:50.694ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14632024-10-17T22:41:50.694ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14642024-10-17T22:41:50.694ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14652024-10-17T22:41:50.694ZDEBGtest_metrics: client response result = Ok(Response { url: "http://[::1]:35099/omicron-zones", status: 204, headers: {"x-request-id": "4a7f80dc-e5f5-402b-9caa-5e32e8067ac3", "date": "Thu, 17 Oct 2024 22:41:50 GMT"} })
14662024-10-17T22:41:50.694ZDEBGtest_metrics: client request body = Some(Body) method = PUT uri = http://[::1]:35099/network-bootstore-config
14672024-10-17T22:41:50.695ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
14682024-10-17T22:41:50.697ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14692024-10-17T22:41:50.697ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14702024-10-17T22:41:50.697ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14712024-10-17T22:41:50.697ZTRACtest_metrics (dropshot): incoming request local_addr = [::1]:35099 method = PUT remote_addr = [::1]:59830 req_id = b48c1ee9-d61d-46ba-9e95-7084cb634cb2 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /network-bootstore-config
14722024-10-17T22:41:50.697ZINFOtest_metrics (dropshot): request completed latency_us = 701 local_addr = [::1]:35099 method = PUT remote_addr = [::1]:59830 req_id = b48c1ee9-d61d-46ba-9e95-7084cb634cb2 response_code = 204 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /network-bootstore-config
14732024-10-17T22:41:50.697ZDEBGtest_metrics: client response result = Ok(Response { url: "http://[::1]:35099/network-bootstore-config", status: 204, headers: {"x-request-id": "b48c1ee9-d61d-46ba-9e95-7084cb634cb2", "date": "Thu, 17 Oct 2024 22:41:50 GMT"} })
14742024-10-17T22:41:50.702ZDEBGtest_metrics (init_with_steps): Running step start_nexus_external
14752024-10-17T22:41:50.702ZDEBGtest_metrics: Starting Nexus (external API)
14762024-10-17T22:41:52.237ZTRACtest_metrics (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
14772024-10-17T22:41:52.237ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
14782024-10-17T22:41:52.238ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
14792024-10-17T22:41:52.238ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
14802024-10-17T22:41:52.239ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
14812024-10-17T22:41:52.240ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14822024-10-17T22:41:52.252ZDEBGtest_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
14832024-10-17T22:41:52.253ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14842024-10-17T22:41:52.253ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14852024-10-17T22:41:52.254ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14862024-10-17T22:41:52.258ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14872024-10-17T22:41:52.262ZDEBGtest_metrics (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
14882024-10-17T22:41:52.263ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
14892024-10-17T22:41:52.263ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
14902024-10-17T22:41:52.264ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
14912024-10-17T22:41:52.269ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
14922024-10-17T22:41:52.269ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
14932024-10-17T22:41:52.270ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
14942024-10-17T22:41:52.284ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14952024-10-17T22:41:52.284ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14962024-10-17T22:41:52.285ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14972024-10-17T22:41:52.288ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14982024-10-17T22:41:52.288ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14992024-10-17T22:41:52.289ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15002024-10-17T22:41:52.315ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15012024-10-17T22:41:52.315ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15022024-10-17T22:41:52.316ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15032024-10-17T22:41:52.318ZINFOtest_metrics (InternalApi): set cockroachdb setting setting = cluster.preserve_downgrade_option value = 22.1
15042024-10-17T22:41:52.319ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15052024-10-17T22:41:52.319ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
15062024-10-17T22:41:52.320ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15072024-10-17T22:41:52.359ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15082024-10-17T22:41:52.359ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15092024-10-17T22:41:52.369ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15102024-10-17T22:41:52.380ZTRACtest_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") }
15112024-10-17T22:41:52.384ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
15122024-10-17T22:41:52.384ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15132024-10-17T22:41:52.385ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15142024-10-17T22:41:52.386ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15152024-10-17T22:41:52.391ZTRACtest_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Rack { parent: Fleet, key: c19a698f-c6f9-4a17-ae30-20d711b8f7dc, lookup_type: ById(c19a698f-c6f9-4a17-ae30-20d711b8f7dc) }
15162024-10-17T22:41:52.392ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
15172024-10-17T22:41:52.392ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15182024-10-17T22:41:52.392ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
15192024-10-17T22:41:52.393ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15202024-10-17T22:41:52.398ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
15212024-10-17T22:41:52.398ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15222024-10-17T22:41:52.398ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15232024-10-17T22:41:52.399ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15242024-10-17T22:41:52.412ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15252024-10-17T22:41:52.428ZDEBGtest_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Rack { parent: Fleet, key: c19a698f-c6f9-4a17-ae30-20d711b8f7dc, lookup_type: ById(c19a698f-c6f9-4a17-ae30-20d711b8f7dc) } result = Ok(())
15262024-10-17T22:41:52.428ZDEBGtest_metrics (InternalApi): updating rack subnet for rack c19a698f-c6f9-4a17-ae30-20d711b8f7dc to Some( V6( Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56, }, ), )
15272024-10-17T22:41:52.428ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15282024-10-17T22:41:52.429ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
15292024-10-17T22:41:52.442ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15302024-10-17T22:41:52.443ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
15312024-10-17T22:41:52.444ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15322024-10-17T22:41:52.444ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15332024-10-17T22:41:52.444ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15342024-10-17T22:41:52.455ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15352024-10-17T22:41:52.464ZDEBGtest_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
15362024-10-17T22:41:52.465ZTRACtest_metrics (BackgroundTasks): successful lookup of siloed resource "VpcSubnet" using built-in user background_task = service_firewall_rule_propagation
15372024-10-17T22:41:52.465ZTRACtest_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") }
15382024-10-17T22:41:52.465ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
15392024-10-17T22:41:52.465ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15402024-10-17T22:41:52.465ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15412024-10-17T22:41:52.465ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15422024-10-17T22:41:52.467ZINFOtest_metrics (InternalApi): Using static configuration for external switchports
15432024-10-17T22:41:52.467ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15442024-10-17T22:41:52.467ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
15452024-10-17T22:41:52.468ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15462024-10-17T22:41:52.478ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
15472024-10-17T22:41:52.478ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15482024-10-17T22:41:52.478ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15492024-10-17T22:41:52.479ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15502024-10-17T22:41:52.485ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
15512024-10-17T22:41:52.485ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15522024-10-17T22:41:52.485ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15532024-10-17T22:41:52.486ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15542024-10-17T22:41:52.497ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15552024-10-17T22:41:52.506ZDEBGtest_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
15562024-10-17T22:41:52.506ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15572024-10-17T22:41:52.506ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15582024-10-17T22:41:52.507ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15592024-10-17T22:41:52.524ZDEBGtest_metrics (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
15602024-10-17T22:41:52.524ZTRACtest_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet
15612024-10-17T22:41:52.524ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
15622024-10-17T22:41:52.524ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15632024-10-17T22:41:52.524ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15642024-10-17T22:41:52.525ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15652024-10-17T22:41:52.530ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15662024-10-17T22:41:52.534ZDEBGtest_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
15672024-10-17T22:41:52.535ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15682024-10-17T22:41:52.535ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15692024-10-17T22:41:52.535ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15702024-10-17T22:41:52.558ZTRACtest_metrics (service-firewall-rule-progator): fetched allowlist from DB allowed = AllowList { id: 001de000-a110-4000-8000-000000000000, time_created: 2024-10-17T22:19:18.973614Z, time_modified: 2024-10-17T22:19:18.973614Z, allowed_ips: None } background_task = service_firewall_rule_propagation
15712024-10-17T22:41:52.559ZDEBGtest_metrics (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
15722024-10-17T22:41:52.560ZDEBGtest_metrics (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = []
15732024-10-17T22:41:52.560ZDEBGtest_metrics (service-firewall-rule-progator): resolved 0 rules for sleds background_task = service_firewall_rule_propagation
15742024-10-17T22:41:52.590ZINFOtest_metrics (InternalApi): Recording Rack Network Configuration
15752024-10-17T22:41:52.590ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15762024-10-17T22:41:52.591ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
15772024-10-17T22:41:52.591ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15782024-10-17T22:41:52.769ZTRACtest_metrics (InternalApi): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
15792024-10-17T22:41:52.769ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
15802024-10-17T22:41:52.769ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15812024-10-17T22:41:52.769ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
15822024-10-17T22:41:52.770ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15832024-10-17T22:41:52.776ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15842024-10-17T22:41:52.779ZDEBGtest_metrics (InternalApi): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
15852024-10-17T22:41:52.779ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15862024-10-17T22:41:52.780ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
15872024-10-17T22:41:52.780ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15882024-10-17T22:41:52.781ZDEBGtest_metrics (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = []
15892024-10-17T22:41:52.781ZDEBGtest_metrics (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
15902024-10-17T22:41:52.781ZINFOtest_metrics (service-firewall-rule-progator): updated firewall rules on 0 sleds background_task = service_firewall_rule_propagation
15912024-10-17T22:41:52.781ZDEBGtest_metrics (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 3.487263672s
15922024-10-17T22:41:52.781ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 3.487313364s iteration = 1 status = {}
15932024-10-17T22:41:52.783ZTRACtest_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = IpPool { parent: Fleet, key: 8e763b22-048b-4258-be28-09bccbb58c17, lookup_type: ByName("oxide-service-pool") }
15942024-10-17T22:41:52.783ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
15952024-10-17T22:41:52.783ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15962024-10-17T22:41:52.783ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
15972024-10-17T22:41:52.784ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15982024-10-17T22:41:52.786ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15992024-10-17T22:41:52.793ZDEBGtest_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = IpPool { parent: Fleet, key: 8e763b22-048b-4258-be28-09bccbb58c17, lookup_type: ByName("oxide-service-pool") } result = Ok(())
16002024-10-17T22:41:52.793ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16012024-10-17T22:41:52.793ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
16022024-10-17T22:41:52.794ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16032024-10-17T22:41:52.796ZTRACtest_metrics (InternalApi): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = IpPool { parent: Fleet, key: 8e763b22-048b-4258-be28-09bccbb58c17, lookup_type: ByName("oxide-service-pool") }
16042024-10-17T22:41:52.796ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16052024-10-17T22:41:52.796ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16062024-10-17T22:41:52.796ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
16072024-10-17T22:41:52.797ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16082024-10-17T22:41:52.800ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16092024-10-17T22:41:52.806ZDEBGtest_metrics (InternalApi): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = IpPool { parent: Fleet, key: 8e763b22-048b-4258-be28-09bccbb58c17, lookup_type: ByName("oxide-service-pool") } result = Ok(())
16102024-10-17T22:41:52.842ZTRACtest_metrics (InternalApi): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = IpPool { parent: Fleet, key: 8e763b22-048b-4258-be28-09bccbb58c17, lookup_type: ByName("oxide-service-pool") }
16112024-10-17T22:41:52.842ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16122024-10-17T22:41:52.842ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16132024-10-17T22:41:52.843ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
16142024-10-17T22:41:52.843ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16152024-10-17T22:41:52.847ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16162024-10-17T22:41:52.850ZDEBGtest_metrics (InternalApi): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = IpPool { parent: Fleet, key: 8e763b22-048b-4258-be28-09bccbb58c17, lookup_type: ByName("oxide-service-pool") } result = Ok(())
16172024-10-17T22:41:52.855ZTRACtest_metrics (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = BlueprintConfig
16182024-10-17T22:41:52.856ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16192024-10-17T22:41:52.856ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16202024-10-17T22:41:52.856ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
16212024-10-17T22:41:52.857ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16222024-10-17T22:41:52.859ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16232024-10-17T22:41:52.863ZDEBGtest_metrics (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = BlueprintConfig result = Ok(())
16242024-10-17T22:41:53.122ZINFOtest_metrics (InternalApi): inserted blueprint blueprint_id = 87401968-edd9-437c-a1ae-135cb455a53e
16252024-10-17T22:41:53.122ZTRACtest_metrics (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = BlueprintConfig
16262024-10-17T22:41:53.123ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16272024-10-17T22:41:53.123ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16282024-10-17T22:41:53.123ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
16292024-10-17T22:41:53.124ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16302024-10-17T22:41:53.130ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16312024-10-17T22:41:53.134ZDEBGtest_metrics (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = BlueprintConfig result = Ok(())
16322024-10-17T22:41:53.140ZINFOtest_metrics (InternalApi): No networking records needed for cockroach_db service
16332024-10-17T22:41:53.140ZINFOtest_metrics (InternalApi): No networking records needed for clickhouse service
16342024-10-17T22:41:53.140ZINFOtest_metrics (InternalApi): No networking records needed for internal_dns service
16352024-10-17T22:41:53.237ZINFOtest_metrics (InternalApi): Inserted networking records for external_dns service
16362024-10-17T22:41:53.258ZINFOtest_metrics (InternalApi): Inserted networking records for nexus service
16372024-10-17T22:41:53.259ZINFOtest_metrics (InternalApi): No networking records needed for crucible_pantry service
16382024-10-17T22:41:53.259ZINFOtest_metrics (InternalApi): Inserted service networking records
16392024-10-17T22:41:53.259ZINFOtest_metrics (InternalApi): Inserted physical disks
16402024-10-17T22:41:53.259ZINFOtest_metrics (InternalApi): Inserted zpools
16412024-10-17T22:41:53.259ZINFOtest_metrics (InternalApi): Inserted datasets
16422024-10-17T22:41:53.301ZINFOtest_metrics (InternalApi): Populated DNS tables for internal DNS
16432024-10-17T22:41:53.306ZINFOtest_metrics (InternalApi): Populated DNS tables for external DNS
16442024-10-17T22:41:53.306ZTRACtest_metrics (InternalApi): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
16452024-10-17T22:41:53.306ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16462024-10-17T22:41:53.306ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16472024-10-17T22:41:53.307ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
16482024-10-17T22:41:53.307ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16492024-10-17T22:41:53.309ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16502024-10-17T22:41:53.314ZDEBGtest_metrics (InternalApi): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
16512024-10-17T22:41:53.314ZTRACtest_metrics (InternalApi): authorize begin action = ModifyPolicy actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
16522024-10-17T22:41:53.314ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16532024-10-17T22:41:53.314ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16542024-10-17T22:41:53.315ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
16552024-10-17T22:41:53.315ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16562024-10-17T22:41:53.317ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16572024-10-17T22:41:53.321ZDEBGtest_metrics (InternalApi): authorize result action = ModifyPolicy actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
16582024-10-17T22:41:53.348ZTRACtest_metrics (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = DnsConfig
16592024-10-17T22:41:53.349ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16602024-10-17T22:41:53.349ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16612024-10-17T22:41:53.349ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
16622024-10-17T22:41:53.353ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16632024-10-17T22:41:53.354ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16642024-10-17T22:41:53.360ZDEBGtest_metrics (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = DnsConfig result = Ok(())
16652024-10-17T22:41:53.360ZTRACtest_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = DnsConfig
16662024-10-17T22:41:53.360ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16672024-10-17T22:41:53.361ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16682024-10-17T22:41:53.361ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
16692024-10-17T22:41:53.361ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16702024-10-17T22:41:53.363ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16712024-10-17T22:41:53.372ZDEBGtest_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = DnsConfig result = Ok(())
16722024-10-17T22:41:53.378ZTRACtest_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = DnsConfig
16732024-10-17T22:41:53.378ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16742024-10-17T22:41:53.379ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16752024-10-17T22:41:53.379ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
16762024-10-17T22:41:53.379ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16772024-10-17T22:41:53.382ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16782024-10-17T22:41:53.390ZDEBGtest_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = DnsConfig result = Ok(())
16792024-10-17T22:41:53.396ZINFOtest_metrics (InternalApi): Created recovery silo
16802024-10-17T22:41:53.398ZINFOtest_metrics (InternalApi): Created recovery user
16812024-10-17T22:41:53.426ZINFOtest_metrics (InternalApi): Created recovery user's password
16822024-10-17T22:41:53.426ZTRACtest_metrics (InternalApi): authorize begin action = ModifyPolicy actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Silo { parent: Fleet, key: 29732a9a-d8c5-46bb-8799-9f9175aa6d42, lookup_type: ById(29732a9a-d8c5-46bb-8799-9f9175aa6d42) }
16832024-10-17T22:41:53.426ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 29732a9a-d8c5-46bb-8799-9f9175aa6d42 resource_type = Silo
16842024-10-17T22:41:53.426ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16852024-10-17T22:41:53.427ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
16862024-10-17T22:41:53.427ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16872024-10-17T22:41:53.433ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16882024-10-17T22:41:53.433ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16892024-10-17T22:41:53.433ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
16902024-10-17T22:41:53.434ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16912024-10-17T22:41:53.443ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16922024-10-17T22:41:53.445ZDEBGtest_metrics (InternalApi): authorize result action = ModifyPolicy actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Silo { parent: Fleet, key: 29732a9a-d8c5-46bb-8799-9f9175aa6d42, lookup_type: ById(29732a9a-d8c5-46bb-8799-9f9175aa6d42) } result = Ok(())
16932024-10-17T22:41:53.445ZDEBGtest_metrics (InternalApi): Generated role assignment queries
16942024-10-17T22:41:53.446ZINFOtest_metrics (InternalApi): Granted Silo privileges
16952024-10-17T22:41:53.447ZTRACtest_metrics (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
16962024-10-17T22:41:53.447ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16972024-10-17T22:41:53.447ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16982024-10-17T22:41:53.447ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
16992024-10-17T22:41:53.448ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17002024-10-17T22:41:53.450ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17012024-10-17T22:41:53.456ZDEBGtest_metrics (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
17022024-10-17T22:41:53.491ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17032024-10-17T22:41:53.499ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17042024-10-17T22:41:53.501ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17052024-10-17T22:41:53.504ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17062024-10-17T22:41:53.504ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17072024-10-17T22:41:53.505ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17082024-10-17T22:41:53.508ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17092024-10-17T22:41:53.508ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17102024-10-17T22:41:53.509ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17112024-10-17T22:41:53.512ZTRACtest_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
17122024-10-17T22:41:53.513ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
17132024-10-17T22:41:53.513ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17142024-10-17T22:41:53.513ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17152024-10-17T22:41:53.513ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17162024-10-17T22:41:53.517ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
17172024-10-17T22:41:53.517ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17182024-10-17T22:41:53.517ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17192024-10-17T22:41:53.518ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17202024-10-17T22:41:53.531ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
17212024-10-17T22:41:53.531ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17222024-10-17T22:41:53.531ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17232024-10-17T22:41:53.531ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17242024-10-17T22:41:53.534ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17252024-10-17T22:41:53.544ZDEBGtest_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
17262024-10-17T22:41:53.544ZTRACtest_metrics (InternalApi): successful lookup of siloed resource "Vpc" using built-in user
17272024-10-17T22:41:53.544ZTRACtest_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
17282024-10-17T22:41:53.544ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
17292024-10-17T22:41:53.544ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17302024-10-17T22:41:53.544ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17312024-10-17T22:41:53.544ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17322024-10-17T22:41:53.546ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
17332024-10-17T22:41:53.546ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17342024-10-17T22:41:53.546ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17352024-10-17T22:41:53.547ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17362024-10-17T22:41:53.549ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
17372024-10-17T22:41:53.549ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17382024-10-17T22:41:53.550ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17392024-10-17T22:41:53.550ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17402024-10-17T22:41:53.551ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17412024-10-17T22:41:53.561ZDEBGtest_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
17422024-10-17T22:41:53.561ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17432024-10-17T22:41:53.561ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17442024-10-17T22:41:53.561ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17452024-10-17T22:41:53.564ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17462024-10-17T22:41:53.564ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17472024-10-17T22:41:53.565ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17482024-10-17T22:41:53.568ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17492024-10-17T22:41:53.569ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17502024-10-17T22:41:53.569ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17512024-10-17T22:41:53.571ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17522024-10-17T22:41:53.572ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17532024-10-17T22:41:53.572ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17542024-10-17T22:41:53.574ZTRACtest_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
17552024-10-17T22:41:53.574ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
17562024-10-17T22:41:53.575ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17572024-10-17T22:41:53.575ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17582024-10-17T22:41:53.575ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17592024-10-17T22:41:53.576ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
17602024-10-17T22:41:53.577ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17612024-10-17T22:41:53.577ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17622024-10-17T22:41:53.577ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17632024-10-17T22:41:53.583ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
17642024-10-17T22:41:53.584ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17652024-10-17T22:41:53.584ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17662024-10-17T22:41:53.585ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17672024-10-17T22:41:53.591ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17682024-10-17T22:41:53.599ZDEBGtest_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
17692024-10-17T22:41:53.599ZTRACtest_metrics (InternalApi): successful lookup of siloed resource "Vpc" using built-in user
17702024-10-17T22:41:53.599ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17712024-10-17T22:41:53.599ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17722024-10-17T22:41:53.599ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17732024-10-17T22:41:53.602ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17742024-10-17T22:41:53.602ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17752024-10-17T22:41:53.603ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17762024-10-17T22:41:53.604ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17772024-10-17T22:41:53.605ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17782024-10-17T22:41:53.606ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17792024-10-17T22:41:53.608ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17802024-10-17T22:41:53.608ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17812024-10-17T22:41:53.609ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17822024-10-17T22:41:53.611ZTRACtest_metrics (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") }
17832024-10-17T22:41:53.612ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
17842024-10-17T22:41:53.612ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17852024-10-17T22:41:53.612ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17862024-10-17T22:41:53.613ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17872024-10-17T22:41:53.615ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
17882024-10-17T22:41:53.615ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17892024-10-17T22:41:53.616ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17902024-10-17T22:41:53.616ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17912024-10-17T22:41:53.619ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
17922024-10-17T22:41:53.619ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17932024-10-17T22:41:53.619ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17942024-10-17T22:41:53.620ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17952024-10-17T22:41:53.621ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17962024-10-17T22:41:53.628ZDEBGtest_metrics (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
17972024-10-17T22:41:53.628ZTRACtest_metrics (InternalApi): successful lookup of siloed resource "VpcSubnet" using built-in user
17982024-10-17T22:41:53.628ZTRACtest_metrics (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") }
17992024-10-17T22:41:53.628ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
18002024-10-17T22:41:53.628ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18012024-10-17T22:41:53.628ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18022024-10-17T22:41:53.629ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18032024-10-17T22:41:53.631ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
18042024-10-17T22:41:53.631ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18052024-10-17T22:41:53.631ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18062024-10-17T22:41:53.632ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18072024-10-17T22:41:53.634ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
18082024-10-17T22:41:53.634ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18092024-10-17T22:41:53.634ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18102024-10-17T22:41:53.634ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18112024-10-17T22:41:53.636ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18122024-10-17T22:41:53.644ZDEBGtest_metrics (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
18132024-10-17T22:41:53.644ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18142024-10-17T22:41:53.644ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18152024-10-17T22:41:53.645ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18162024-10-17T22:41:53.652ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18172024-10-17T22:41:53.653ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18182024-10-17T22:41:53.653ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18192024-10-17T22:41:53.656ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18202024-10-17T22:41:53.656ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18212024-10-17T22:41:53.657ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18222024-10-17T22:41:53.659ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18232024-10-17T22:41:53.660ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18242024-10-17T22:41:53.660ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18252024-10-17T22:41:53.661ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18262024-10-17T22:41:53.661ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18272024-10-17T22:41:53.662ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18282024-10-17T22:41:53.664ZTRACtest_metrics (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") }
18292024-10-17T22:41:53.664ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
18302024-10-17T22:41:53.664ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18312024-10-17T22:41:53.664ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18322024-10-17T22:41:53.665ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18332024-10-17T22:41:53.666ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
18342024-10-17T22:41:53.667ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18352024-10-17T22:41:53.667ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18362024-10-17T22:41:53.667ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18372024-10-17T22:41:53.668ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
18382024-10-17T22:41:53.669ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18392024-10-17T22:41:53.669ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18402024-10-17T22:41:53.669ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18412024-10-17T22:41:53.670ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18422024-10-17T22:41:53.678ZDEBGtest_metrics (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
18432024-10-17T22:41:53.678ZTRACtest_metrics (InternalApi): successful lookup of siloed resource "VpcSubnet" using built-in user
18442024-10-17T22:41:53.678ZTRACtest_metrics (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") }
18452024-10-17T22:41:53.678ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
18462024-10-17T22:41:53.678ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18472024-10-17T22:41:53.678ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18482024-10-17T22:41:53.679ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18492024-10-17T22:41:53.681ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
18502024-10-17T22:41:53.681ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18512024-10-17T22:41:53.681ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18522024-10-17T22:41:53.682ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18532024-10-17T22:41:53.683ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
18542024-10-17T22:41:53.683ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18552024-10-17T22:41:53.683ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18562024-10-17T22:41:53.684ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18572024-10-17T22:41:53.687ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18582024-10-17T22:41:53.696ZDEBGtest_metrics (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
18592024-10-17T22:41:53.696ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18602024-10-17T22:41:53.696ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18612024-10-17T22:41:53.696ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18622024-10-17T22:41:53.702ZDEBGtest_metrics (nexus): resolved names for firewall rules instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: a95f8ebc-23b6-4c41-9f4c-951b45ca6452, kind: Service { id: 017ff6a2-3b72-4882-9b0b-97601068f99c }, name: Name("nexus-017ff6a2-3b72-4882-9b0b-97601068f99c"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 1])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("external-dns"): [NetworkInterface { id: a0119282-face-4af2-8569-41fd572ff751, kind: Service { id: 0e0c52d8-7b87-4b83-9828-05f7d187ffb1 }, name: Name("external-dns-0e0c52d8-7b87-4b83-9828-05f7d187ffb1"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 0])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
18632024-10-17T22:41:53.702ZTRACtest_metrics (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
18642024-10-17T22:41:53.702ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
18652024-10-17T22:41:53.702ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18662024-10-17T22:41:53.702ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18672024-10-17T22:41:53.703ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18682024-10-17T22:41:53.704ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18692024-10-17T22:41:53.717ZDEBGtest_metrics (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
18702024-10-17T22:41:53.717ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18712024-10-17T22:41:53.717ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18722024-10-17T22:41:53.718ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18732024-10-17T22:41:53.725ZTRACtest_metrics (nexus): fetched allowlist from DB allowed = AllowList { id: 001de000-a110-4000-8000-000000000000, time_created: 2024-10-17T22:19:18.973614Z, time_modified: 2024-10-17T22:41:53.456678Z, allowed_ips: None }
18742024-10-17T22:41:53.725ZDEBGtest_metrics (nexus): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified.
18752024-10-17T22:41:53.726ZDEBGtest_metrics (nexus): resolved firewall rules for sled agents sled_agent_rules = [ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: a0119282-face-4af2-8569-41fd572ff751, kind: Service { id: 0e0c52d8-7b87-4b83-9828-05f7d187ffb1 }, name: Name("external-dns-0e0c52d8-7b87-4b83-9828-05f7d187ffb1"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 0])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }, ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: a95f8ebc-23b6-4c41-9f4c-951b45ca6452, kind: Service { id: 017ff6a2-3b72-4882-9b0b-97601068f99c }, name: Name("nexus-017ff6a2-3b72-4882-9b0b-97601068f99c"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 1])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }]
18762024-10-17T22:41:53.727ZDEBGtest_metrics (nexus): resolved 2 rules for sleds
18772024-10-17T22:41:53.831ZDEBGtest_metrics (nexus): resolved sleds for vpc oxide-services vpc_to_sled = [Sled { identity: SledIdentity { id: b6d65341-167c-41df-9b5c-41cded99c229, time_created: 2024-10-17T22:41:49.344227Z, time_modified: 2024-10-17T22:41:49.344227Z }, time_deleted: None, rcgen: Generation(Generation(1)), rack_id: c19a698f-c6f9-4a17-ae30-20d711b8f7dc, is_scrimlet: true, serial_number: "sim-b6d65341", part_number: "Unknown", revision: SqlU32(0), usable_hardware_threads: SqlU32(16), usable_physical_ram: ByteCount(ByteCount(34359738368)), reservoir_size: ByteCount(ByteCount(17179869184)), ip: ::1, port: SqlU16(40953), last_used_address: ::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)) }]
18782024-10-17T22:41:53.831ZDEBGtest_metrics (nexus): sending firewall rules to sled agents
18792024-10-17T22:41:53.831ZTRACtest_metrics (InstanceAllocator): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database
18802024-10-17T22:41:53.831ZDEBGtest_metrics (InstanceAllocator): roles roles = RoleSet { roles: {} }
18812024-10-17T22:41:53.831ZDEBGtest_metrics (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
18822024-10-17T22:41:53.834ZTRACtest_metrics (InstanceAllocator): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Sled { parent: Fleet, key: b6d65341-167c-41df-9b5c-41cded99c229, lookup_type: ById(b6d65341-167c-41df-9b5c-41cded99c229) }
18832024-10-17T22:41:53.835ZTRACtest_metrics (InstanceAllocator): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
18842024-10-17T22:41:53.835ZTRACtest_metrics (InstanceAllocator): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database
18852024-10-17T22:41:53.835ZDEBGtest_metrics (InstanceAllocator): roles roles = RoleSet { roles: {} }
18862024-10-17T22:41:53.835ZDEBGtest_metrics (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
18872024-10-17T22:41:53.838ZDEBGtest_metrics (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
18882024-10-17T22:41:53.843ZDEBGtest_metrics (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Sled { parent: Fleet, key: b6d65341-167c-41df-9b5c-41cded99c229, lookup_type: ById(b6d65341-167c-41df-9b5c-41cded99c229) } result = Ok(())
18892024-10-17T22:41:53.880ZDEBGtest_metrics (nexus): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 body = Some(Body) method = PUT uri = http://[::1]:40953/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
18902024-10-17T22:41:53.881ZINFOtest_metrics (dropshot): accepted connection local_addr = [::1]:40953 remote_addr = [::1]:57528 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
18912024-10-17T22:41:53.881ZTRACtest_metrics (dropshot): incoming request local_addr = [::1]:40953 method = PUT remote_addr = [::1]:57528 req_id = 1d6f5958-5428-4cb8-a3d5-35834a003cbb sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
18922024-10-17T22:41:53.882ZINFOtest_metrics (dropshot): request completed latency_us = 632 local_addr = [::1]:40953 method = PUT remote_addr = [::1]:57528 req_id = 1d6f5958-5428-4cb8-a3d5-35834a003cbb response_code = 204 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
18932024-10-17T22:41:53.882ZDEBGtest_metrics (nexus): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 result = Ok(Response { url: "http://[::1]:40953/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "1d6f5958-5428-4cb8-a3d5-35834a003cbb", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
18942024-10-17T22:41:53.883ZINFOtest_metrics (nexus): updated firewall rules on 1 sleds
18952024-10-17T22:41:53.884ZTRACtest_metrics (ServiceBalancer): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database
18962024-10-17T22:41:53.884ZDEBGtest_metrics (ServiceBalancer): roles roles = RoleSet { roles: {} }
18972024-10-17T22:41:53.884ZDEBGtest_metrics (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
18982024-10-17T22:41:53.885ZDEBGtest_metrics (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 2 reason = Signaled
18992024-10-17T22:41:53.885ZDEBGtest_metrics (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
19002024-10-17T22:41:53.885ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
19012024-10-17T22:41:53.885ZDEBGtest_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
19022024-10-17T22:41:53.885ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
19032024-10-17T22:41:53.885ZDEBGtest_metrics (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 2 reason = Signaled
19042024-10-17T22:41:53.886ZTRACtest_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal
19052024-10-17T22:41:53.886ZDEBGtest_metrics (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 2 reason = Signaled
19062024-10-17T22:41:53.886ZDEBGtest_metrics (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
19072024-10-17T22:41:53.887ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
19082024-10-17T22:41:53.887ZDEBGtest_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
19092024-10-17T22:41:53.887ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
19102024-10-17T22:41:53.887ZDEBGtest_metrics (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 2 reason = Signaled
19112024-10-17T22:41:53.887ZTRACtest_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal
19122024-10-17T22:41:53.888ZDEBGtest_metrics (BackgroundTasks): activating background_task = external_endpoints iteration = 2 reason = Signaled
19132024-10-17T22:41:53.888ZTRACtest_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet
19142024-10-17T22:41:53.888ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = external_endpoints resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19152024-10-17T22:41:53.888ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
19162024-10-17T22:41:53.888ZDEBGtest_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
19172024-10-17T22:41:53.888ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
19182024-10-17T22:41:53.888ZDEBGtest_metrics (BackgroundTasks): activating background_task = inventory_collection iteration = 4 reason = Signaled
19192024-10-17T22:41:53.888ZTRACtest_metrics (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
19202024-10-17T22:41:53.888ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19212024-10-17T22:41:53.888ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
19222024-10-17T22:41:53.888ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
19232024-10-17T22:41:53.889ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
19242024-10-17T22:41:53.889ZTRACtest_metrics (dns): buffer buffer = Length: 75 (0x4b) bytes\n0000: de 55 01 00 00 01 00 00 00 00 00 01 0c 5f 6e 61 .U..........._na\n0010: 6d 65 73 65 72 76 69 63 65 04 5f 74 63 70 0d 63 meservice._tcp.c\n0020: 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 ontrol-plane.oxi\n0030: 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 de.internal..!..\n0040: 00 00 29 04 d0 00 00 00 00 00 00 ..)........ kind = dns peer_addr = [::1]:61612 req_id = 33747466-7a8d-44da-baef-8376d8cebfcd
19252024-10-17T22:41:53.889ZDEBGtest_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 56917,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_nameservice._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_nameservice._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 12,\n 95,\n 110,\n 97,\n 109,\n 101,\n 115,\n 101,\n 114,\n 118,\n 105,\n 99,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:61612 req_id = 33747466-7a8d-44da-baef-8376d8cebfcd
19262024-10-17T22:41:53.890ZDEBGtest_metrics (store): query key key = _nameservice._tcp
19272024-10-17T22:41:53.890ZDEBGtest_metrics (store): query key key = 996eea1f-0c18-450c-8228-39db7dc3685a.host
19282024-10-17T22:41:53.890ZDEBGtest_metrics (dns): dns response additional_records = [Record { name_labels: Name("996eea1f-0c18-450c-8228-39db7dc3685a.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:61612 query = LowerQuery { name: LowerName(Name("_nameservice._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 41599, target: Name("996eea1f-0c18-450c-8228-39db7dc3685a.host.control-plane.oxide.internal") })) }] req_id = 33747466-7a8d-44da-baef-8376d8cebfcd
19292024-10-17T22:41:53.890ZTRACtest_metrics (dns): buffer buffer = Length: 76 (0x4c) bytes\n0000: 4c 37 01 00 00 01 00 00 00 00 00 01 0d 5f 65 78 L7..........._ex\n0010: 74 65 72 6e 61 6c 2d 64 6e 73 04 5f 74 63 70 0d ternal-dns._tcp.\n0020: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0030: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 21 00 ide.internal..!.\n0040: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:63670 req_id = 366e751f-3eec-4f33-823b-8ba028b39d3c
19302024-10-17T22:41:53.890ZDEBGtest_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 19511,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_external-dns._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_external-dns._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 13,\n 95,\n 101,\n 120,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 45,\n 100,\n 110,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63670 req_id = 366e751f-3eec-4f33-823b-8ba028b39d3c
19312024-10-17T22:41:53.890ZDEBGtest_metrics (store): query key key = _external-dns._tcp
19322024-10-17T22:41:53.890ZDEBGtest_metrics (store): query key key = 0e0c52d8-7b87-4b83-9828-05f7d187ffb1.host
19332024-10-17T22:41:53.890ZDEBGtest_metrics (dns): dns response additional_records = [Record { name_labels: Name("0e0c52d8-7b87-4b83-9828-05f7d187ffb1.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:63670 query = LowerQuery { name: LowerName(Name("_external-dns._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 36177, target: Name("0e0c52d8-7b87-4b83-9828-05f7d187ffb1.host.control-plane.oxide.internal") })) }] req_id = 366e751f-3eec-4f33-823b-8ba028b39d3c
19342024-10-17T22:41:53.890ZDEBGtest_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 41599, target: Name("996eea1f-0c18-450c-8228-39db7dc3685a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("996eea1f-0c18-450c-8228-39db7dc3685a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2085, tv_nsec: 659485140 } })
19352024-10-17T22:41:53.890ZTRACtest_metrics (DnsResolver): lookup_service_targets: looking up SRV target
19362024-10-17T22:41:53.891ZDEBGtest_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 36177, target: Name("0e0c52d8-7b87-4b83-9828-05f7d187ffb1.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("0e0c52d8-7b87-4b83-9828-05f7d187ffb1.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2085, tv_nsec: 659780347 } })
19372024-10-17T22:41:53.891ZTRACtest_metrics (DnsResolver): lookup_service_targets: looking up SRV target
19382024-10-17T22:41:53.891ZTRACtest_metrics (dns): buffer buffer = Length: 99 (0x63) bytes\n0000: 20 76 01 00 00 01 00 00 00 00 00 01 24 39 39 36 v..........$996\n0010: 65 65 61 31 66 2d 30 63 31 38 2d 34 35 30 63 2d eea1f-0c18-450c-\n0020: 38 32 32 38 2d 33 39 64 62 37 64 63 33 36 38 35 8228-39db7dc3685\n0030: 61 04 68 6f 73 74 0d 63 6f 6e 74 72 6f 6c 2d 70 a.host.control-p\n0040: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0050: 6e 61 6c 00 00 1c 00 01 00 00 29 04 d0 00 00 00 nal.......).....\n0060: 00 00 00 ... kind = dns peer_addr = [::1]:63470 req_id = 0499f317-98ac-4a6f-9ba9-8ec4d60c3654
19392024-10-17T22:41:53.894ZDEBGtest_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 8310,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("996eea1f-0c18-450c-8228-39db7dc3685a.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("996eea1f-0c18-450c-8228-39db7dc3685a.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 36,\n 57,\n 57,\n 54,\n 101,\n 101,\n 97,\n 49,\n 102,\n 45,\n 48,\n 99,\n 49,\n 56,\n 45,\n 52,\n 53,\n 48,\n 99,\n 45,\n 56,\n 50,\n 50,\n 56,\n 45,\n 51,\n 57,\n 100,\n 98,\n 55,\n 100,\n 99,\n 51,\n 54,\n 56,\n 53,\n 97,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63470 req_id = 0499f317-98ac-4a6f-9ba9-8ec4d60c3654
19402024-10-17T22:41:53.894ZDEBGtest_metrics (store): query key key = 996eea1f-0c18-450c-8228-39db7dc3685a.host
19412024-10-17T22:41:53.894ZDEBGtest_metrics (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:63470 query = LowerQuery { name: LowerName(Name("996eea1f-0c18-450c-8228-39db7dc3685a.host.control-plane.oxide.internal.")), original: Query { name: Name("996eea1f-0c18-450c-8228-39db7dc3685a.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("996eea1f-0c18-450c-8228-39db7dc3685a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 0499f317-98ac-4a6f-9ba9-8ec4d60c3654
19422024-10-17T22:41:53.894ZTRACtest_metrics (dns): buffer buffer = Length: 99 (0x63) bytes\n0000: bd af 01 00 00 01 00 00 00 00 00 01 24 30 65 30 ............$0e0\n0010: 63 35 32 64 38 2d 37 62 38 37 2d 34 62 38 33 2d c52d8-7b87-4b83-\n0020: 39 38 32 38 2d 30 35 66 37 64 31 38 37 66 66 62 9828-05f7d187ffb\n0030: 31 04 68 6f 73 74 0d 63 6f 6e 74 72 6f 6c 2d 70 1.host.control-p\n0040: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0050: 6e 61 6c 00 00 1c 00 01 00 00 29 04 d0 00 00 00 nal.......).....\n0060: 00 00 00 ... kind = dns peer_addr = [::1]:49836 req_id = 928d896d-293b-499d-8df9-6e0b93c2c94e
19432024-10-17T22:41:53.894ZDEBGtest_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 48559,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("0e0c52d8-7b87-4b83-9828-05f7d187ffb1.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("0e0c52d8-7b87-4b83-9828-05f7d187ffb1.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 36,\n 48,\n 101,\n 48,\n 99,\n 53,\n 50,\n 100,\n 56,\n 45,\n 55,\n 98,\n 56,\n 55,\n 45,\n 52,\n 98,\n 56,\n 51,\n 45,\n 57,\n 56,\n 50,\n 56,\n 45,\n 48,\n 53,\n 102,\n 55,\n 100,\n 49,\n 56,\n 55,\n 102,\n 102,\n 98,\n 49,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:49836 req_id = 928d896d-293b-499d-8df9-6e0b93c2c94e
19442024-10-17T22:41:53.895ZDEBGtest_metrics (store): query key key = 0e0c52d8-7b87-4b83-9828-05f7d187ffb1.host
19452024-10-17T22:41:53.895ZDEBGtest_metrics (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:49836 query = LowerQuery { name: LowerName(Name("0e0c52d8-7b87-4b83-9828-05f7d187ffb1.host.control-plane.oxide.internal.")), original: Query { name: Name("0e0c52d8-7b87-4b83-9828-05f7d187ffb1.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("0e0c52d8-7b87-4b83-9828-05f7d187ffb1.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 928d896d-293b-499d-8df9-6e0b93c2c94e
19462024-10-17T22:41:53.895ZINFOtest_metrics (BackgroundTasks): found DNS servers (initial) addresses = DnsServersList { addresses: [[::1]:41599] } background_task = dns_servers_internal dns_group = internal
19472024-10-17T22:41:53.895ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 6.557818ms iteration = 2 status = {"addresses":["[::1]:41599"]}
19482024-10-17T22:41:53.896ZINFOtest_metrics (BackgroundTasks): found DNS servers (initial) addresses = DnsServersList { addresses: [[::1]:36177] } background_task = dns_servers_external dns_group = external
19492024-10-17T22:41:53.896ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 5.859291ms iteration = 2 status = {"addresses":["[::1]:36177"]}
19502024-10-17T22:41:53.896ZDEBGtest_metrics (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 2 reason = Dependency
19512024-10-17T22:41:53.896ZWARNtest_metrics (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_internal dns_group = internal reason = no config
19522024-10-17T22:41:53.896ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 22.431\u{b5}s iteration = 2 status = {"error":"no config"}
19532024-10-17T22:41:53.896ZDEBGtest_metrics (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 2 reason = Dependency
19542024-10-17T22:41:53.896ZWARNtest_metrics (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_external dns_group = external reason = no config
19552024-10-17T22:41:53.897ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 10.421\u{b5}s iteration = 2 status = {"error":"no config"}
19562024-10-17T22:41:53.898ZTRACtest_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig
19572024-10-17T22:41:53.898ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_internal dns_group = internal resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19582024-10-17T22:41:53.898ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
19592024-10-17T22:41:53.899ZDEBGtest_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
19602024-10-17T22:41:53.899ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
19612024-10-17T22:41:53.899ZTRACtest_metrics (ServiceBalancer): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: c19a698f-c6f9-4a17-ae30-20d711b8f7dc, lookup_type: ById(c19a698f-c6f9-4a17-ae30-20d711b8f7dc) }
19622024-10-17T22:41:53.899ZTRACtest_metrics (ServiceBalancer): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19632024-10-17T22:41:53.899ZTRACtest_metrics (ServiceBalancer): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database
19642024-10-17T22:41:53.900ZDEBGtest_metrics (ServiceBalancer): roles roles = RoleSet { roles: {} }
19652024-10-17T22:41:53.900ZDEBGtest_metrics (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
19662024-10-17T22:41:53.900ZTRACtest_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig
19672024-10-17T22:41:53.900ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_external dns_group = external resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19682024-10-17T22:41:53.900ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
19692024-10-17T22:41:53.900ZDEBGtest_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
19702024-10-17T22:41:53.900ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
19712024-10-17T22:41:53.900ZDEBGtest_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19722024-10-17T22:41:53.904ZDEBGtest_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
19732024-10-17T22:41:53.904ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
19742024-10-17T22:41:53.904ZDEBGtest_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
19752024-10-17T22:41:53.905ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
19762024-10-17T22:41:53.906ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19772024-10-17T22:41:53.914ZDEBGtest_metrics (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
19782024-10-17T22:41:53.914ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
19792024-10-17T22:41:53.914ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
19802024-10-17T22:41:53.914ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
19812024-10-17T22:41:53.916ZDEBGtest_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19822024-10-17T22:41:53.925ZDEBGtest_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
19832024-10-17T22:41:53.926ZDEBGtest_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19842024-10-17T22:41:53.934ZDEBGtest_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
19852024-10-17T22:41:53.939ZTRACtest_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig
19862024-10-17T22:41:53.939ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = external_endpoints resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19872024-10-17T22:41:53.939ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
19882024-10-17T22:41:53.939ZDEBGtest_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
19892024-10-17T22:41:53.939ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
19902024-10-17T22:41:53.940ZDEBGtest_metrics (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = []
19912024-10-17T22:41:53.940ZTRACtest_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
19922024-10-17T22:41:53.955ZDEBGtest_metrics (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
19932024-10-17T22:41:53.956ZDEBGtest_metrics (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
19942024-10-17T22:41:53.956ZDEBGtest_metrics (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
19952024-10-17T22:41:53.956ZTRACtest_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig
19962024-10-17T22:41:53.956ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_internal dns_group = internal resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19972024-10-17T22:41:53.956ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
19982024-10-17T22:41:53.956ZDEBGtest_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
19992024-10-17T22:41:53.956ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
20002024-10-17T22:41:53.959ZDEBGtest_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20012024-10-17T22:41:53.975ZDEBGtest_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
20022024-10-17T22:41:53.975ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
20032024-10-17T22:41:53.975ZDEBGtest_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
20042024-10-17T22:41:53.976ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
20052024-10-17T22:41:53.977ZDEBGtest_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20062024-10-17T22:41:53.983ZDEBGtest_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
20072024-10-17T22:41:53.991ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
20082024-10-17T22:41:53.991ZDEBGtest_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
20092024-10-17T22:41:53.992ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
20102024-10-17T22:41:53.992ZDEBGtest_metrics (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
20112024-10-17T22:41:53.992ZDEBGtest_metrics (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
20122024-10-17T22:41:53.992ZDEBGtest_metrics (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
20132024-10-17T22:41:53.992ZTRACtest_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig
20142024-10-17T22:41:53.993ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_external dns_group = external resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
20152024-10-17T22:41:53.993ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
20162024-10-17T22:41:53.993ZDEBGtest_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
20172024-10-17T22:41:53.993ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
20182024-10-17T22:41:53.999ZTRACtest_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet
20192024-10-17T22:41:54.000ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = external_endpoints resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
20202024-10-17T22:41:54.000ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
20212024-10-17T22:41:54.000ZDEBGtest_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
20222024-10-17T22:41:54.001ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
20232024-10-17T22:41:54.002ZDEBGtest_metrics (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20242024-10-17T22:41:54.009ZDEBGtest_metrics (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: c19a698f-c6f9-4a17-ae30-20d711b8f7dc, lookup_type: ById(c19a698f-c6f9-4a17-ae30-20d711b8f7dc) } result = Ok(())
20252024-10-17T22:41:54.009ZINFOtest_metrics (nexus): Rack initialized
20262024-10-17T22:41:54.009ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
20272024-10-17T22:41:54.009ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
20282024-10-17T22:41:54.014ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
20292024-10-17T22:41:54.015ZTRACtest_metrics (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 93 07 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:60141 req_id = 5f64e04e-4ba0-44ef-b968-c6d75b3342b8
20302024-10-17T22:41:54.015ZDEBGtest_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 37639,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60141 req_id = 5f64e04e-4ba0-44ef-b968-c6d75b3342b8
20312024-10-17T22:41:54.016ZDEBGtest_metrics (store): query key key = _mgs._tcp
20322024-10-17T22:41:54.016ZDEBGtest_metrics (store): query key key = dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host
20332024-10-17T22:41:54.022ZDEBGtest_metrics (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
20342024-10-17T22:41:54.023ZDEBGtest_metrics (dns): dns response additional_records = [Record { name_labels: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:60141 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 33937, target: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal") })) }, Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 44607, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 5f64e04e-4ba0-44ef-b968-c6d75b3342b8
20352024-10-17T22:41:54.024ZDEBGtest_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 33937, target: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 44607, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2085, tv_nsec: 793243197 } })
20362024-10-17T22:41:54.026ZTRACtest_metrics (DnsResolver): lookup_service_targets: looking up SRV target
20372024-10-17T22:41:54.027ZTRACtest_metrics (DnsResolver): lookup_service_targets: looking up SRV target
20382024-10-17T22:41:54.028ZDEBGtest_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20392024-10-17T22:41:54.041ZDEBGtest_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
20402024-10-17T22:41:54.041ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
20412024-10-17T22:41:54.041ZDEBGtest_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
20422024-10-17T22:41:54.042ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
20432024-10-17T22:41:54.042ZDEBGtest_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20442024-10-17T22:41:54.051ZDEBGtest_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
20452024-10-17T22:41:54.051ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
20462024-10-17T22:41:54.051ZDEBGtest_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
20472024-10-17T22:41:54.052ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
20482024-10-17T22:41:54.054ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
20492024-10-17T22:41:54.054ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
20502024-10-17T22:41:54.055ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
20512024-10-17T22:41:54.055ZTRACtest_metrics (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 6f 05 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e o...........-den\n0010: 64 72 69 74 65 2d 30 33 39 62 65 35 36 30 2d 35 drite-039be560-5\n0020: 34 63 63 2d 34 39 65 33 2d 38 38 64 66 2d 31 61 4cc-49e3-88df-1a\n0030: 32 39 64 61 64 62 66 39 31 33 04 68 6f 73 74 0d 29dadbf913.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:59237 req_id = 928bd5ee-71ac-4e50-914f-255eb87788d8
20522024-10-17T22:41:54.056ZDEBGtest_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 28421,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 48,\n 51,\n 57,\n 98,\n 101,\n 53,\n 54,\n 48,\n 45,\n 53,\n 52,\n 99,\n 99,\n 45,\n 52,\n 57,\n 101,\n 51,\n 45,\n 56,\n 56,\n 100,\n 102,\n 45,\n 49,\n 97,\n 50,\n 57,\n 100,\n 97,\n 100,\n 98,\n 102,\n 57,\n 49,\n 51,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:59237 req_id = 928bd5ee-71ac-4e50-914f-255eb87788d8
20532024-10-17T22:41:54.056ZDEBGtest_metrics (store): query key key = dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host
20542024-10-17T22:41:54.056ZDEBGtest_metrics (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:59237 query = LowerQuery { name: LowerName(Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 928bd5ee-71ac-4e50-914f-255eb87788d8
20552024-10-17T22:41:54.057ZTRACtest_metrics (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 0d 94 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:49257 req_id = b3d8987d-7731-409d-bf20-834f21cba7f8
20562024-10-17T22:41:54.057ZDEBGtest_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 3476,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:49257 req_id = b3d8987d-7731-409d-bf20-834f21cba7f8
20572024-10-17T22:41:54.057ZDEBGtest_metrics (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
20582024-10-17T22:41:54.057ZDEBGtest_metrics (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:49257 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = b3d8987d-7731-409d-bf20-834f21cba7f8
20592024-10-17T22:41:54.213ZTRACtest_metrics (DnsResolver): lookup_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
20602024-10-17T22:41:54.215ZDEBGtest_metrics (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
20612024-10-17T22:41:54.215ZDEBGtest_metrics (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 2907eb7b-cfa4-44c5-a6f6-96e1e2e4fa6f dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
20622024-10-17T22:41:54.215ZTRACtest_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig
20632024-10-17T22:41:54.215ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_internal dns_group = internal resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
20642024-10-17T22:41:54.215ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
20652024-10-17T22:41:54.216ZDEBGtest_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
20662024-10-17T22:41:54.216ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
20672024-10-17T22:41:54.216ZDEBGtest_metrics (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
20682024-10-17T22:41:54.216ZDEBGtest_metrics (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = f7bca40a-4301-43a3-b7a5-fe7b01c00551 dns_zone_name = oxide-dev.test found_so_far = 0 version = 2
20692024-10-17T22:41:54.216ZTRACtest_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig
20702024-10-17T22:41:54.216ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_external dns_group = external resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
20712024-10-17T22:41:54.217ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
20722024-10-17T22:41:54.217ZDEBGtest_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
20732024-10-17T22:41:54.217ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
20742024-10-17T22:41:54.217ZINFOtest_metrics (BackgroundTasks): found Silo/DNS/TLS config (initial) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide-dev.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-10-17T22:41:47.111093Z, time_modified: 2024-10-17T22:41:47.111093Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "test-suite-silo.sys.oxide-dev.test": ExternalEndpoint { silo_id: 29732a9a-d8c5-46bb-8799-9f9175aa6d42, db_silo: Silo { identity: SiloIdentity { id: 29732a9a-d8c5-46bb-8799-9f9175aa6d42, name: Name(Name("test-suite-silo")), description: "built-in recovery Silo", time_created: 2024-10-17T22:41:53.306448Z, time_modified: 2024-10-17T22:41:53.306448Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(1)) }, tls_certs: [] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide-dev.test" }, NoSiloCerts { silo_id: 29732a9a-d8c5-46bb-8799-9f9175aa6d42, dns_name: "test-suite-silo.sys.oxide-dev.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 29732a9a-d8c5-46bb-8799-9f9175aa6d42, db_silo: Silo { identity: SiloIdentity { id: 29732a9a-d8c5-46bb-8799-9f9175aa6d42, name: Name(Name("test-suite-silo")), description: "built-in recovery Silo", time_created: 2024-10-17T22:41:53.306448Z, time_modified: 2024-10-17T22:41:53.306448Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(1)) }, tls_certs: [] }) }
20752024-10-17T22:41:54.218ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 330.814947ms iteration = 2 status = {"by_dns_name":{"default-silo.sys.oxide-dev.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"test-suite-silo.sys.oxide-dev.test":{"silo_id":"29732a9a-d8c5-46bb-8799-9f9175aa6d42","tls_certs":[]}},"default_endpoint":{"silo_id":"29732a9a-d8c5-46bb-8799-9f9175aa6d42","tls_certs":[]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide-dev.test\\" has no usable certificates","silo 29732a9a-d8c5-46bb-8799-9f9175aa6d42 with DNS name \\"test-suite-silo.sys.oxide-dev.test\\" has no usable certificates"]}
20762024-10-17T22:41:54.223ZTRACtest_metrics (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 37 f3 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c 7............_cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:59478 req_id = 37c55e97-109e-4195-b151-9b41f8a0a66f
20772024-10-17T22:41:54.223ZDEBGtest_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 14323,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:59478 req_id = 37c55e97-109e-4195-b151-9b41f8a0a66f
20782024-10-17T22:41:54.225ZDEBGtest_metrics (store): query key key = _clickhouse-admin-keeper._tcp
20792024-10-17T22:41:54.226ZERROtest_metrics (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 14323, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:59478 req_id = 37c55e97-109e-4195-b151-9b41f8a0a66f
20802024-10-17T22:41:54.226ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
20812024-10-17T22:41:54.226ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
20822024-10-17T22:41:54.227ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
20832024-10-17T22:41:54.227ZDEBGtest_metrics (BackgroundTasks): begin collection background_task = inventory_collection
20842024-10-17T22:41:54.227ZDEBGtest_metrics (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:33937
20852024-10-17T22:41:54.227ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/ignition
20862024-10-17T22:41:54.236ZDEBGtest_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20872024-10-17T22:41:54.245ZDEBGtest_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
20882024-10-17T22:41:54.245ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
20892024-10-17T22:41:54.245ZDEBGtest_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
20902024-10-17T22:41:54.246ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
20912024-10-17T22:41:54.248ZDEBGtest_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20922024-10-17T22:41:54.253ZDEBGtest_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
20932024-10-17T22:41:54.253ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
20942024-10-17T22:41:54.253ZDEBGtest_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
20952024-10-17T22:41:54.253ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
20962024-10-17T22:41:54.257ZTRACtest_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
20972024-10-17T22:41:54.257ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
20982024-10-17T22:41:54.257ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
20992024-10-17T22:41:54.257ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
21002024-10-17T22:41:54.257ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21012024-10-17T22:41:54.258ZDEBGtest_metrics (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 2907eb7b-cfa4-44c5-a6f6-96e1e2e4fa6f dns_zone_name = control-plane.oxide.internal found_so_far = 18 version = 1
21022024-10-17T22:41:54.259ZDEBGtest_metrics (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
21032024-10-17T22:41:54.259ZINFOtest_metrics (BackgroundTasks): found latest generation (first find) background_task = dns_config_internal dns_group = internal generation = 1
21042024-10-17T22:41:54.259ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 374.103277ms iteration = 2 status = {"generation":1}
21052024-10-17T22:41:54.268ZDEBGtest_metrics (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 3 reason = Dependency
21062024-10-17T22:41:54.308ZDEBGtest_metrics (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[::1]:41599 generation = 1 method = PUT servers = DnsServersList { addresses: [[::1]:41599] } uri = http://[::1]:41599/config
21072024-10-17T22:41:54.310ZDEBGtest_metrics (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = f7bca40a-4301-43a3-b7a5-fe7b01c00551 dns_zone_name = oxide-dev.test found_so_far = 1 version = 2
21082024-10-17T22:41:54.311ZDEBGtest_metrics (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
21092024-10-17T22:41:54.311ZINFOtest_metrics (BackgroundTasks): found latest generation (first find) background_task = dns_config_external dns_group = external generation = 2
21102024-10-17T22:41:54.311ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 423.827834ms iteration = 2 status = {"generation":2}
21112024-10-17T22:41:54.311ZDEBGtest_metrics (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 3 reason = Dependency
21122024-10-17T22:41:54.345ZDEBGtest_metrics (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[::1]:36177 generation = 2 method = PUT servers = DnsServersList { addresses: [[::1]:36177] } uri = http://[::1]:36177/config
21132024-10-17T22:41:54.347ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
21142024-10-17T22:41:54.347ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21152024-10-17T22:41:54.347ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
21162024-10-17T22:41:54.347ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21172024-10-17T22:41:54.348ZINFOtest_metrics (http): accepted connection kind = dns local_addr = [::1]:41599 remote_addr = [::1]:52940
21182024-10-17T22:41:54.349ZINFOtest_metrics (http): accepted connection kind = dns local_addr = [::1]:36177 remote_addr = [::1]:48044
21192024-10-17T22:41:54.353ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
21202024-10-17T22:41:54.353ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21212024-10-17T22:41:54.353ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
21222024-10-17T22:41:54.354ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21232024-10-17T22:41:54.354ZTRACtest_metrics (http): incoming request kind = dns local_addr = [::1]:41599 method = PUT remote_addr = [::1]:52940 req_id = bcbc3237-b8b0-4b90-ae07-f9fdcdd9c8fc uri = /config
21242024-10-17T22:41:54.354ZTRACtest_metrics (http): incoming request kind = dns local_addr = [::1]:36177 method = PUT remote_addr = [::1]:48044 req_id = 79036533-0408-4710-8cf1-e91ad68b6286 uri = /config
21252024-10-17T22:41:54.356ZINFOtest_metrics (store): attempting generation update new_generation = 1 req_id = bcbc3237-b8b0-4b90-ae07-f9fdcdd9c8fc
21262024-10-17T22:41:54.356ZINFOtest_metrics (store): updated generation new_generation = 1 req_id = bcbc3237-b8b0-4b90-ae07-f9fdcdd9c8fc
21272024-10-17T22:41:54.358ZINFOtest_metrics (store): attempting generation update new_generation = 2 req_id = 79036533-0408-4710-8cf1-e91ad68b6286
21282024-10-17T22:41:54.359ZINFOtest_metrics (store): pruning trees for generations newer than 0
21292024-10-17T22:41:54.359ZDEBGtest_metrics (store): creating tree tree_name = generation_2_zone_oxide-dev.test
21302024-10-17T22:41:54.359ZINFOtest_metrics (http): request completed kind = dns latency_us = 4822 local_addr = [::1]:41599 method = PUT remote_addr = [::1]:52940 req_id = bcbc3237-b8b0-4b90-ae07-f9fdcdd9c8fc response_code = 204 uri = /config
21312024-10-17T22:41:54.365ZDEBGtest_metrics (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[::1]:41599 generation = 1 result = Ok(Response { url: "http://[::1]:41599/config", status: 204, headers: {"x-request-id": "bcbc3237-b8b0-4b90-ae07-f9fdcdd9c8fc", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} }) servers = DnsServersList { addresses: [[::1]:41599] }
21322024-10-17T22:41:54.365ZINFOtest_metrics (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[::1]:41599 generation = 1 servers = DnsServersList { addresses: [[::1]:41599] }
21332024-10-17T22:41:54.366ZINFOtest_metrics (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal generation = 1 servers = DnsServersList { addresses: [[::1]:41599] }
21342024-10-17T22:41:54.366ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 98.080769ms iteration = 3 status = {"generation":1,"server_results":{"[::1]:41599":{"Ok":null}}}
21352024-10-17T22:41:54.366ZDEBGtest_metrics (store): updating current config
21362024-10-17T22:41:54.367ZDEBGtest_metrics (store): flushing default tree
21372024-10-17T22:41:54.367ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21382024-10-17T22:41:54.376ZDEBGtest_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
21392024-10-17T22:41:54.376ZTRACtest_metrics (InternalApi): successful lookup of siloed resource "Vpc" using built-in user
21402024-10-17T22:41:54.376ZTRACtest_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
21412024-10-17T22:41:54.376ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
21422024-10-17T22:41:54.376ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21432024-10-17T22:41:54.377ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
21442024-10-17T22:41:54.377ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21452024-10-17T22:41:54.380ZINFOtest_metrics (store): pruning trees for generations older than 2 keep = 3
21462024-10-17T22:41:54.380ZINFOtest_metrics (store): updated generation new_generation = 2 req_id = 79036533-0408-4710-8cf1-e91ad68b6286
21472024-10-17T22:41:54.380ZINFOtest_metrics (http): request completed kind = dns latency_us = 24347 local_addr = [::1]:36177 method = PUT remote_addr = [::1]:48044 req_id = 79036533-0408-4710-8cf1-e91ad68b6286 response_code = 204 uri = /config
21482024-10-17T22:41:54.380ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
21492024-10-17T22:41:54.380ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21502024-10-17T22:41:54.380ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
21512024-10-17T22:41:54.380ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21522024-10-17T22:41:54.383ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "1210ae22-e202-4f0e-b634-8db4c0e83e1b", "content-length": "835", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
21532024-10-17T22:41:54.386ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0
21542024-10-17T22:41:54.388ZDEBGtest_metrics (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[::1]:36177 generation = 2 result = Ok(Response { url: "http://[::1]:36177/config", status: 204, headers: {"x-request-id": "79036533-0408-4710-8cf1-e91ad68b6286", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} }) servers = DnsServersList { addresses: [[::1]:36177] }
21552024-10-17T22:41:54.388ZINFOtest_metrics (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[::1]:36177 generation = 2 servers = DnsServersList { addresses: [[::1]:36177] }
21562024-10-17T22:41:54.388ZINFOtest_metrics (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external generation = 2 servers = DnsServersList { addresses: [[::1]:36177] }
21572024-10-17T22:41:54.388ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 74.610965ms iteration = 3 status = {"generation":2,"server_results":{"[::1]:36177":{"Ok":null}}}
21582024-10-17T22:41:54.389ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
21592024-10-17T22:41:54.390ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21602024-10-17T22:41:54.390ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
21612024-10-17T22:41:54.390ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21622024-10-17T22:41:54.395ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21632024-10-17T22:41:54.402ZDEBGtest_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
21642024-10-17T22:41:54.402ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21652024-10-17T22:41:54.402ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
21662024-10-17T22:41:54.403ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21672024-10-17T22:41:54.412ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e9ea431a-0cae-4ff2-9a90-b83a7e973477", "content-length": "734", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
21682024-10-17T22:41:54.412ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0/component/sp/caboose?firmware_slot=0
21692024-10-17T22:41:54.414ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21702024-10-17T22:41:54.414ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
21712024-10-17T22:41:54.414ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21722024-10-17T22:41:54.422ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21732024-10-17T22:41:54.422ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
21742024-10-17T22:41:54.422ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21752024-10-17T22:41:54.424ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21762024-10-17T22:41:54.424ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
21772024-10-17T22:41:54.425ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21782024-10-17T22:41:54.427ZTRACtest_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
21792024-10-17T22:41:54.427ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
21802024-10-17T22:41:54.427ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21812024-10-17T22:41:54.427ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
21822024-10-17T22:41:54.427ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21832024-10-17T22:41:54.428ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f360d629-3df2-4474-90c5-75fd0c0da7d3", "content-length": "111", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
21842024-10-17T22:41:54.429ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0/component/sp/caboose?firmware_slot=1
21852024-10-17T22:41:54.429ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
21862024-10-17T22:41:54.429ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21872024-10-17T22:41:54.432ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
21882024-10-17T22:41:54.432ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21892024-10-17T22:41:54.432ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
21902024-10-17T22:41:54.432ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21912024-10-17T22:41:54.432ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
21922024-10-17T22:41:54.432ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21932024-10-17T22:41:54.433ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c65fe8ba-1018-43b6-a061-c17a45333fc0", "content-length": "111", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
21942024-10-17T22:41:54.433ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0/component/rot/caboose?firmware_slot=0
21952024-10-17T22:41:54.434ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21962024-10-17T22:41:54.442ZDEBGtest_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
21972024-10-17T22:41:54.442ZTRACtest_metrics (InternalApi): successful lookup of siloed resource "Vpc" using built-in user
21982024-10-17T22:41:54.442ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21992024-10-17T22:41:54.447ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22002024-10-17T22:41:54.448ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22012024-10-17T22:41:54.448ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d1c2f8c3-5304-46d5-8f27-0ff73b5e7acc", "content-length": "167", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
22022024-10-17T22:41:54.448ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0/component/rot/caboose?firmware_slot=1
22032024-10-17T22:41:54.449ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22042024-10-17T22:41:54.449ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22052024-10-17T22:41:54.449ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22062024-10-17T22:41:54.449ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22072024-10-17T22:41:54.449ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22082024-10-17T22:41:54.449ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22092024-10-17T22:41:54.449ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "db27dd30-9c36-4e6a-8b56-f29d335fb25f", "content-length": "167", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
22102024-10-17T22:41:54.449ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0/component/stage0/caboose?firmware_slot=0
22112024-10-17T22:41:54.450ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22122024-10-17T22:41:54.461ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22132024-10-17T22:41:54.462ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22142024-10-17T22:41:54.462ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "eaa254dd-e15d-4ad8-9293-ad9a00ce738f", "content-length": "175", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
22152024-10-17T22:41:54.462ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0/component/stage0/caboose?firmware_slot=1
22162024-10-17T22:41:54.462ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9dfb46ea-cd14-481e-a2f5-42e657f0a543", "content-length": "175", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
22172024-10-17T22:41:54.462ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0/component/rot/cmpa
22182024-10-17T22:41:54.462ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3ba550cd-c00c-4c2c-8549-a5da98e56ff6", "content-length": "702", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
22192024-10-17T22:41:54.462ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0/component/rot/cfpa
22202024-10-17T22:41:54.462ZTRACtest_metrics (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") }
22212024-10-17T22:41:54.462ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
22222024-10-17T22:41:54.462ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22232024-10-17T22:41:54.462ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22242024-10-17T22:41:54.462ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22252024-10-17T22:41:54.462ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fd105bab-e264-436a-b026-fd11fd77dbb8", "content-length": "727", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
22262024-10-17T22:41:54.463ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0/component/rot/cfpa
22272024-10-17T22:41:54.463ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0ac0f806-dd1b-4806-9725-e56b046df8db", "content-length": "729", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
22282024-10-17T22:41:54.463ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0/component/rot/cfpa
22292024-10-17T22:41:54.463ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
22302024-10-17T22:41:54.463ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22312024-10-17T22:41:54.463ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22322024-10-17T22:41:54.463ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22332024-10-17T22:41:54.463ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "46829037-9526-436c-a0f5-e96ffc1fdb83", "content-length": "728", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
22342024-10-17T22:41:54.463ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1
22352024-10-17T22:41:54.463ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a0e56340-30e9-48bf-bdb6-ff03756b2a20", "content-length": "734", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
22362024-10-17T22:41:54.463ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1/component/sp/caboose?firmware_slot=0
22372024-10-17T22:41:54.466ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1c364b0d-9757-42b7-b1a0-b9aee0a75de1", "content-length": "111", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
22382024-10-17T22:41:54.468ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1/component/sp/caboose?firmware_slot=1
22392024-10-17T22:41:54.468ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
22402024-10-17T22:41:54.468ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22412024-10-17T22:41:54.468ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22422024-10-17T22:41:54.468ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22432024-10-17T22:41:54.468ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "defebc39-4c4a-4103-bc0d-abb1a046dfd6", "content-length": "111", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
22442024-10-17T22:41:54.471ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1/component/rot/caboose?firmware_slot=0
22452024-10-17T22:41:54.471ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22462024-10-17T22:41:54.481ZDEBGtest_metrics (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
22472024-10-17T22:41:54.482ZTRACtest_metrics (InternalApi): successful lookup of siloed resource "VpcSubnet" using built-in user
22482024-10-17T22:41:54.482ZTRACtest_metrics (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") }
22492024-10-17T22:41:54.482ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
22502024-10-17T22:41:54.482ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22512024-10-17T22:41:54.483ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22522024-10-17T22:41:54.484ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22532024-10-17T22:41:54.484ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b71592e4-c734-4701-9b3e-5cd3fadf405b", "content-length": "167", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
22542024-10-17T22:41:54.484ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1/component/rot/caboose?firmware_slot=1
22552024-10-17T22:41:54.486ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
22562024-10-17T22:41:54.486ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22572024-10-17T22:41:54.486ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22582024-10-17T22:41:54.486ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22592024-10-17T22:41:54.486ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
22602024-10-17T22:41:54.486ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22612024-10-17T22:41:54.486ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22622024-10-17T22:41:54.487ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22632024-10-17T22:41:54.487ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e61b8a34-5440-4317-a285-c5ad8eeecddd", "content-length": "167", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
22642024-10-17T22:41:54.487ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1/component/stage0/caboose?firmware_slot=0
22652024-10-17T22:41:54.489ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22662024-10-17T22:41:54.497ZDEBGtest_metrics (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
22672024-10-17T22:41:54.497ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22682024-10-17T22:41:54.497ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22692024-10-17T22:41:54.497ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22702024-10-17T22:41:54.504ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e216935f-a4a6-41f1-86cc-c98099de4e87", "content-length": "175", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
22712024-10-17T22:41:54.504ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1/component/stage0/caboose?firmware_slot=1
22722024-10-17T22:41:54.504ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8aaf0f9a-3dc4-411e-98fd-3d6e4c4b20b9", "content-length": "175", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
22732024-10-17T22:41:54.504ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1/component/rot/cmpa
22742024-10-17T22:41:54.504ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a8b7298f-ed83-4e25-a139-6e9bb37ecb7a", "content-length": "702", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
22752024-10-17T22:41:54.504ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1/component/rot/cfpa
22762024-10-17T22:41:54.506ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22772024-10-17T22:41:54.506ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22782024-10-17T22:41:54.506ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22792024-10-17T22:41:54.506ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b179f524-31b2-42b2-aa02-c028455c2fba", "content-length": "727", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
22802024-10-17T22:41:54.506ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1/component/rot/cfpa
22812024-10-17T22:41:54.506ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e495db3f-f3f6-459b-b842-3ca4974ef2cd", "content-length": "729", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
22822024-10-17T22:41:54.506ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1/component/rot/cfpa
22832024-10-17T22:41:54.506ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "08045b23-11e6-4d37-b77c-09b8d1d0fc7d", "content-length": "728", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
22842024-10-17T22:41:54.507ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0
22852024-10-17T22:41:54.511ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7ca58134-5bae-4fcc-b6f1-ae3540de4fd3", "content-length": "723", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
22862024-10-17T22:41:54.511ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0/component/sp/caboose?firmware_slot=0
22872024-10-17T22:41:54.511ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ce702c50-0782-4148-b57c-26ba86bcdc70", "content-length": "109", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
22882024-10-17T22:41:54.511ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0/component/sp/caboose?firmware_slot=1
22892024-10-17T22:41:54.511ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22902024-10-17T22:41:54.511ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22912024-10-17T22:41:54.511ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22922024-10-17T22:41:54.513ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7f6058d3-f764-4be6-9e46-c60e02d64a78", "content-length": "109", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
22932024-10-17T22:41:54.513ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0/component/rot/caboose?firmware_slot=0
22942024-10-17T22:41:54.515ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22952024-10-17T22:41:54.515ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22962024-10-17T22:41:54.516ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22972024-10-17T22:41:54.518ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7c9dce5c-e1a7-4d0e-85bf-e322f2a47550", "content-length": "169", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
22982024-10-17T22:41:54.518ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0/component/rot/caboose?firmware_slot=1
22992024-10-17T22:41:54.519ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8418248a-bc13-4403-97df-9b2b10cd1a4c", "content-length": "169", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
23002024-10-17T22:41:54.526ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0/component/stage0/caboose?firmware_slot=0
23012024-10-17T22:41:54.526ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23022024-10-17T22:41:54.526ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
23032024-10-17T22:41:54.526ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23042024-10-17T22:41:54.526ZTRACtest_metrics (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") }
23052024-10-17T22:41:54.527ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
23062024-10-17T22:41:54.527ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23072024-10-17T22:41:54.527ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
23082024-10-17T22:41:54.528ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23092024-10-17T22:41:54.528ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e9806161-b041-4cbd-9b56-f4f223f7387f", "content-length": "178", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
23102024-10-17T22:41:54.528ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0/component/stage0/caboose?firmware_slot=1
23112024-10-17T22:41:54.528ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
23122024-10-17T22:41:54.528ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23132024-10-17T22:41:54.528ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
23142024-10-17T22:41:54.528ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23152024-10-17T22:41:54.528ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8162dc1e-3e6c-432c-82bd-c20ff35481c8", "content-length": "178", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
23162024-10-17T22:41:54.528ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0/component/rot/cmpa
23172024-10-17T22:41:54.528ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f24557e2-f202-41f6-aa51-8d0a47603956", "content-length": "702", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
23182024-10-17T22:41:54.528ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0/component/rot/cfpa
23192024-10-17T22:41:54.528ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
23202024-10-17T22:41:54.530ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23212024-10-17T22:41:54.530ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
23222024-10-17T22:41:54.530ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23232024-10-17T22:41:54.530ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "255f5ee7-e79c-4c2f-8ef0-7b82b938bac6", "content-length": "727", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
23242024-10-17T22:41:54.530ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0/component/rot/cfpa
23252024-10-17T22:41:54.530ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23262024-10-17T22:41:54.539ZDEBGtest_metrics (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
23272024-10-17T22:41:54.539ZTRACtest_metrics (InternalApi): successful lookup of siloed resource "VpcSubnet" using built-in user
23282024-10-17T22:41:54.539ZTRACtest_metrics (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") }
23292024-10-17T22:41:54.540ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
23302024-10-17T22:41:54.540ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23312024-10-17T22:41:54.540ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
23322024-10-17T22:41:54.540ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23332024-10-17T22:41:54.540ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4411ef8e-dae4-4f19-a4a6-a547e71fc1d6", "content-length": "729", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
23342024-10-17T22:41:54.542ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0/component/rot/cfpa
23352024-10-17T22:41:54.542ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd0ea31f-1d1c-4477-baea-dfcd05cf6e8a", "content-length": "728", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
23362024-10-17T22:41:54.542ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1
23372024-10-17T22:41:54.542ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
23382024-10-17T22:41:54.542ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23392024-10-17T22:41:54.542ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
23402024-10-17T22:41:54.542ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23412024-10-17T22:41:54.544ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cdaa0ff9-058e-4dbb-8f8e-08a8b2d32096", "content-length": "723", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
23422024-10-17T22:41:54.546ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1/component/sp/caboose?firmware_slot=0
23432024-10-17T22:41:54.546ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
23442024-10-17T22:41:54.546ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23452024-10-17T22:41:54.546ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
23462024-10-17T22:41:54.546ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23472024-10-17T22:41:54.547ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23482024-10-17T22:41:54.558ZDEBGtest_metrics (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
23492024-10-17T22:41:54.559ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23502024-10-17T22:41:54.559ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
23512024-10-17T22:41:54.559ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23522024-10-17T22:41:54.559ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a67636ba-eb86-44b0-9a12-8d6af8979157", "content-length": "109", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
23532024-10-17T22:41:54.559ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1/component/sp/caboose?firmware_slot=1
23542024-10-17T22:41:54.563ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ed47eac8-7b32-44e4-9073-5b302679d652", "content-length": "109", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
23552024-10-17T22:41:54.564ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1/component/rot/caboose?firmware_slot=0
23562024-10-17T22:41:54.566ZDEBGtest_metrics (InternalApi): resolved names for firewall rules instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: a0119282-face-4af2-8569-41fd572ff751, kind: Service { id: 0e0c52d8-7b87-4b83-9828-05f7d187ffb1 }, name: Name("external-dns-0e0c52d8-7b87-4b83-9828-05f7d187ffb1"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 0])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: a95f8ebc-23b6-4c41-9f4c-951b45ca6452, kind: Service { id: 017ff6a2-3b72-4882-9b0b-97601068f99c }, name: Name("nexus-017ff6a2-3b72-4882-9b0b-97601068f99c"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 1])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
23572024-10-17T22:41:54.566ZTRACtest_metrics (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
23582024-10-17T22:41:54.567ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
23592024-10-17T22:41:54.567ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23602024-10-17T22:41:54.567ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
23612024-10-17T22:41:54.568ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23622024-10-17T22:41:54.571ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23632024-10-17T22:41:54.575ZDEBGtest_metrics (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
23642024-10-17T22:41:54.576ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23652024-10-17T22:41:54.577ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
23662024-10-17T22:41:54.577ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23672024-10-17T22:41:54.578ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bda97715-1ba6-405c-9107-5b051b89fac8", "content-length": "169", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
23682024-10-17T22:41:54.578ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1/component/rot/caboose?firmware_slot=1
23692024-10-17T22:41:54.579ZTRACtest_metrics (InternalApi): fetched allowlist from DB allowed = AllowList { id: 001de000-a110-4000-8000-000000000000, time_created: 2024-10-17T22:19:18.973614Z, time_modified: 2024-10-17T22:41:53.456678Z, allowed_ips: None }
23702024-10-17T22:41:54.579ZDEBGtest_metrics (InternalApi): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified.
23712024-10-17T22:41:54.579ZDEBGtest_metrics (InternalApi): resolved firewall rules for sled agents sled_agent_rules = [ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: a0119282-face-4af2-8569-41fd572ff751, kind: Service { id: 0e0c52d8-7b87-4b83-9828-05f7d187ffb1 }, name: Name("external-dns-0e0c52d8-7b87-4b83-9828-05f7d187ffb1"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 0])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }, ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: a95f8ebc-23b6-4c41-9f4c-951b45ca6452, kind: Service { id: 017ff6a2-3b72-4882-9b0b-97601068f99c }, name: Name("nexus-017ff6a2-3b72-4882-9b0b-97601068f99c"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 1])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }]
23722024-10-17T22:41:54.579ZDEBGtest_metrics (InternalApi): resolved 2 rules for sleds
23732024-10-17T22:41:54.582ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e97c8c1d-163d-41cc-82cb-6123efcc2c02", "content-length": "169", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
23742024-10-17T22:41:54.582ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1/component/stage0/caboose?firmware_slot=0
23752024-10-17T22:41:54.585ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "89a696f9-5a41-4709-951f-7db3abc32394", "content-length": "178", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
23762024-10-17T22:41:54.585ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1/component/stage0/caboose?firmware_slot=1
23772024-10-17T22:41:54.587ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0e5a9215-d37e-4832-b3cf-c9a5ae1bdeb9", "content-length": "178", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
23782024-10-17T22:41:54.587ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1/component/rot/cmpa
23792024-10-17T22:41:54.588ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ea380275-e1ce-41d9-9226-9e2a358679d7", "content-length": "702", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
23802024-10-17T22:41:54.589ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1/component/rot/cfpa
23812024-10-17T22:41:54.590ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4a1726d9-8505-454c-90d2-ac15750df780", "content-length": "727", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
23822024-10-17T22:41:54.590ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1/component/rot/cfpa
23832024-10-17T22:41:54.591ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "40ecf984-2289-4abe-b848-427a8a84e70f", "content-length": "729", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
23842024-10-17T22:41:54.592ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1/component/rot/cfpa
23852024-10-17T22:41:54.595ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f9fb8b8b-6f46-4aa9-abac-d0b22a33e1e4", "content-length": "728", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
23862024-10-17T22:41:54.599ZDEBGtest_metrics (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:44607
23872024-10-17T22:41:54.599ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:44607 method = GET uri = http://[::1]:44607/ignition
23882024-10-17T22:41:54.599ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:44607 result = Ok(Response { url: "http://[::1]:44607/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "4baa88e4-97cf-4e91-94be-2747325a18e0", "content-length": "835", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
23892024-10-17T22:41:54.599ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:44607 method = GET uri = http://[::1]:44607/sp/switch/0
23902024-10-17T22:41:54.599ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:44607 result = Ok(Response { url: "http://[::1]:44607/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0c273025-12c4-4a36-9f7f-b1284fb09b2b", "content-length": "734", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
23912024-10-17T22:41:54.599ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:44607 method = GET uri = http://[::1]:44607/sp/switch/1
23922024-10-17T22:41:54.602ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:44607 result = Ok(Response { url: "http://[::1]:44607/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "28b6b21e-b514-4f9d-821a-405bcfb39a39", "content-length": "734", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
23932024-10-17T22:41:54.602ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:44607 method = GET uri = http://[::1]:44607/sp/sled/0
23942024-10-17T22:41:54.604ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:44607 result = Ok(Response { url: "http://[::1]:44607/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "91676062-b2ae-4089-9ca8-2878b601970e", "content-length": "723", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
23952024-10-17T22:41:54.609ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:44607 method = GET uri = http://[::1]:44607/sp/sled/1
23962024-10-17T22:41:54.609ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:44607 result = Ok(Response { url: "http://[::1]:44607/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0bacd14a-28a4-4799-b3bf-6641dade255d", "content-length": "723", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
23972024-10-17T22:41:54.610ZTRACtest_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
23982024-10-17T22:41:54.610ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
23992024-10-17T22:41:54.610ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
24002024-10-17T22:41:54.610ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
24012024-10-17T22:41:54.610ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
24022024-10-17T22:41:54.610ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24032024-10-17T22:41:54.616ZDEBGtest_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
24042024-10-17T22:41:54.617ZTRACtest_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
24052024-10-17T22:41:54.618ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
24062024-10-17T22:41:54.618ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
24072024-10-17T22:41:54.618ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
24082024-10-17T22:41:54.618ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
24092024-10-17T22:41:54.623ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24102024-10-17T22:41:54.629ZDEBGtest_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
24112024-10-17T22:41:54.629ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
24122024-10-17T22:41:54.629ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
24132024-10-17T22:41:54.629ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
24142024-10-17T22:41:54.642ZDEBGtest_metrics (InternalApi): resolved sleds for vpc oxide-services vpc_to_sled = [Sled { identity: SledIdentity { id: b6d65341-167c-41df-9b5c-41cded99c229, time_created: 2024-10-17T22:41:49.344227Z, time_modified: 2024-10-17T22:41:49.344227Z }, time_deleted: None, rcgen: Generation(Generation(1)), rack_id: c19a698f-c6f9-4a17-ae30-20d711b8f7dc, is_scrimlet: true, serial_number: "sim-b6d65341", part_number: "Unknown", revision: SqlU32(0), usable_hardware_threads: SqlU32(16), usable_physical_ram: ByteCount(ByteCount(34359738368)), reservoir_size: ByteCount(ByteCount(17179869184)), ip: ::1, port: SqlU16(40953), last_used_address: ::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)) }]
24152024-10-17T22:41:54.642ZDEBGtest_metrics (InternalApi): sending firewall rules to sled agents
24162024-10-17T22:41:54.642ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
24172024-10-17T22:41:54.642ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
24182024-10-17T22:41:54.644ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
24192024-10-17T22:41:54.647ZTRACtest_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Sled { parent: Fleet, key: b6d65341-167c-41df-9b5c-41cded99c229, lookup_type: ById(b6d65341-167c-41df-9b5c-41cded99c229) }
24202024-10-17T22:41:54.647ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
24212024-10-17T22:41:54.647ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
24222024-10-17T22:41:54.647ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
24232024-10-17T22:41:54.648ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
24242024-10-17T22:41:54.649ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24252024-10-17T22:41:54.657ZDEBGtest_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Sled { parent: Fleet, key: b6d65341-167c-41df-9b5c-41cded99c229, lookup_type: ById(b6d65341-167c-41df-9b5c-41cded99c229) } result = Ok(())
24262024-10-17T22:41:54.694ZDEBGtest_metrics (InternalApi): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 body = Some(Body) method = PUT uri = http://[::1]:40953/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
24272024-10-17T22:41:54.732ZDEBGtest_metrics (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:35099
24282024-10-17T22:41:54.732ZDEBGtest_metrics (BackgroundTasks): client request SledAgent = http://[::1]:35099 background_task = inventory_collection body = None method = GET uri = http://[::1]:35099/inventory
24292024-10-17T22:41:54.732ZINFOtest_metrics (dropshot): accepted connection local_addr = [::1]:40953 remote_addr = [::1]:57530 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
24302024-10-17T22:41:54.733ZINFOtest_metrics (dropshot): accepted connection local_addr = [::1]:35099 remote_addr = [::1]:59832 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
24312024-10-17T22:41:54.733ZTRACtest_metrics (dropshot): incoming request local_addr = [::1]:35099 method = GET remote_addr = [::1]:59832 req_id = e13ffdd0-9195-4bbe-811f-087326af55ba sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /inventory
24322024-10-17T22:41:54.734ZINFOtest_metrics (dropshot): request completed latency_us = 524 local_addr = [::1]:35099 method = GET remote_addr = [::1]:59832 req_id = e13ffdd0-9195-4bbe-811f-087326af55ba response_code = 200 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /inventory
24332024-10-17T22:41:54.734ZTRACtest_metrics (dropshot): incoming request local_addr = [::1]:40953 method = PUT remote_addr = [::1]:57530 req_id = fe26e771-1d71-4035-8250-d01e74de7988 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
24342024-10-17T22:41:54.734ZDEBGtest_metrics (BackgroundTasks): client response SledAgent = http://[::1]:35099 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:35099/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "e13ffdd0-9195-4bbe-811f-087326af55ba", "content-length": "4199", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
24352024-10-17T22:41:54.769ZDEBGtest_metrics (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:40953
24362024-10-17T22:41:54.769ZDEBGtest_metrics (BackgroundTasks): client request SledAgent = http://[::1]:40953 background_task = inventory_collection body = None method = GET uri = http://[::1]:40953/inventory
24372024-10-17T22:41:54.770ZINFOtest_metrics (dropshot): request completed latency_us = 36004 local_addr = [::1]:40953 method = PUT remote_addr = [::1]:57530 req_id = fe26e771-1d71-4035-8250-d01e74de7988 response_code = 204 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
24382024-10-17T22:41:54.770ZINFOtest_metrics (dropshot): accepted connection local_addr = [::1]:40953 remote_addr = [::1]:57532 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
24392024-10-17T22:41:54.771ZDEBGtest_metrics (InternalApi): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 result = Ok(Response { url: "http://[::1]:40953/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "fe26e771-1d71-4035-8250-d01e74de7988", "date": "Thu, 17 Oct 2024 22:41:53 GMT"} })
24402024-10-17T22:41:54.772ZINFOtest_metrics (InternalApi): updated firewall rules on 1 sleds
24412024-10-17T22:41:54.772ZINFOtest_metrics (nexus): plumbed initial IP allowlist
24422024-10-17T22:41:54.810ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /
24432024-10-17T22:41:54.810ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /access
24442024-10-17T22:41:54.810ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /assets/{path}
24452024-10-17T22:41:54.810ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /device/auth
24462024-10-17T22:41:54.810ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /device/confirm
24472024-10-17T22:41:54.810ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /device/success
24482024-10-17T22:41:54.811ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /device/token
24492024-10-17T22:41:54.811ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /device/verify
24502024-10-17T22:41:54.811ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /experimental/v1/probes
24512024-10-17T22:41:54.811ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /experimental/v1/probes
24522024-10-17T22:41:54.811ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = DELETE path = /experimental/v1/probes/{probe}
24532024-10-17T22:41:54.811ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /experimental/v1/probes/{probe}
24542024-10-17T22:41:54.811ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /images
24552024-10-17T22:41:54.811ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /login
24562024-10-17T22:41:54.811ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /login/{silo_name}/local
24572024-10-17T22:41:54.812ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /login/{silo_name}/saml/{provider_name}
24582024-10-17T22:41:54.812ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /login/{silo_name}/saml/{provider_name}
24592024-10-17T22:41:54.812ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /login/{silo_name}/saml/{provider_name}/redirect
24602024-10-17T22:41:54.812ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /lookup/{path}
24612024-10-17T22:41:54.812ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /projects/{path}
24622024-10-17T22:41:54.812ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /projects-new
24632024-10-17T22:41:54.813ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /settings/{path}
24642024-10-17T22:41:54.813ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /system/{path}
24652024-10-17T22:41:54.813ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /utilization
24662024-10-17T22:41:54.813ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/certificates
24672024-10-17T22:41:54.813ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/certificates
24682024-10-17T22:41:54.813ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = DELETE path = /v1/certificates/{certificate}
24692024-10-17T22:41:54.813ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/certificates/{certificate}
24702024-10-17T22:41:54.813ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/disks
24712024-10-17T22:41:54.813ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/disks
24722024-10-17T22:41:54.813ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = DELETE path = /v1/disks/{disk}
24732024-10-17T22:41:54.813ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/disks/{disk}
24742024-10-17T22:41:54.813ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/disks/{disk}/bulk-write
24752024-10-17T22:41:54.813ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/disks/{disk}/bulk-write-start
24762024-10-17T22:41:54.814ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/disks/{disk}/bulk-write-stop
24772024-10-17T22:41:54.814ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/disks/{disk}/finalize
24782024-10-17T22:41:54.814ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/disks/{disk}/metrics/{metric}
24792024-10-17T22:41:54.814ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/floating-ips
24802024-10-17T22:41:54.814ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/floating-ips
24812024-10-17T22:41:54.815ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = DELETE path = /v1/floating-ips/{floating_ip}
24822024-10-17T22:41:54.815ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/floating-ips/{floating_ip}
24832024-10-17T22:41:54.815ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = PUT path = /v1/floating-ips/{floating_ip}
24842024-10-17T22:41:54.815ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/floating-ips/{floating_ip}/attach
24852024-10-17T22:41:54.815ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/floating-ips/{floating_ip}/detach
24862024-10-17T22:41:54.815ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/groups
24872024-10-17T22:41:54.815ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/groups/{group_id}
24882024-10-17T22:41:54.815ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/images
24892024-10-17T22:41:54.816ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/images
24902024-10-17T22:41:54.816ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = DELETE path = /v1/images/{image}
24912024-10-17T22:41:54.816ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/images/{image}
24922024-10-17T22:41:54.816ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/images/{image}/demote
24932024-10-17T22:41:54.816ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/images/{image}/promote
24942024-10-17T22:41:54.816ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/instances
24952024-10-17T22:41:54.816ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/instances
24962024-10-17T22:41:54.816ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = DELETE path = /v1/instances/{instance}
24972024-10-17T22:41:54.816ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/instances/{instance}
24982024-10-17T22:41:54.816ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = PUT path = /v1/instances/{instance}
24992024-10-17T22:41:54.816ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/instances/{instance}/disks
25002024-10-17T22:41:54.816ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/instances/{instance}/disks/attach
25012024-10-17T22:41:54.817ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/instances/{instance}/disks/detach
25022024-10-17T22:41:54.817ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/instances/{instance}/external-ips
25032024-10-17T22:41:54.817ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = DELETE path = /v1/instances/{instance}/external-ips/ephemeral
25042024-10-17T22:41:54.818ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/instances/{instance}/external-ips/ephemeral
25052024-10-17T22:41:54.818ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/instances/{instance}/reboot
25062024-10-17T22:41:54.818ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/instances/{instance}/serial-console
25072024-10-17T22:41:54.818ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/instances/{instance}/serial-console/stream
25082024-10-17T22:41:54.818ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/instances/{instance}/ssh-public-keys
25092024-10-17T22:41:54.818ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/instances/{instance}/start
25102024-10-17T22:41:54.818ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/instances/{instance}/stop
25112024-10-17T22:41:54.818ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/internet-gateway-ip-addresses
25122024-10-17T22:41:54.818ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/internet-gateway-ip-addresses
25132024-10-17T22:41:54.818ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = DELETE path = /v1/internet-gateway-ip-addresses/{address}
25142024-10-17T22:41:54.818ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/internet-gateway-ip-pools
25152024-10-17T22:41:54.818ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/internet-gateway-ip-pools
25162024-10-17T22:41:54.818ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = DELETE path = /v1/internet-gateway-ip-pools/{pool}
25172024-10-17T22:41:54.818ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/internet-gateways
25182024-10-17T22:41:54.818ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/internet-gateways
25192024-10-17T22:41:54.818ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = DELETE path = /v1/internet-gateways/{gateway}
25202024-10-17T22:41:54.818ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/internet-gateways/{gateway}
25212024-10-17T22:41:54.819ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/ip-pools
25222024-10-17T22:41:54.819ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/ip-pools/{pool}
25232024-10-17T22:41:54.819ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/login/{silo_name}/local
25242024-10-17T22:41:54.819ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/logout
25252024-10-17T22:41:54.820ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/me
25262024-10-17T22:41:54.820ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/me/groups
25272024-10-17T22:41:54.820ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/me/ssh-keys
25282024-10-17T22:41:54.820ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/me/ssh-keys
25292024-10-17T22:41:54.820ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = DELETE path = /v1/me/ssh-keys/{ssh_key}
25302024-10-17T22:41:54.820ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/me/ssh-keys/{ssh_key}
25312024-10-17T22:41:54.820ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/metrics/{metric_name}
25322024-10-17T22:41:54.820ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/network-interfaces
25332024-10-17T22:41:54.820ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/network-interfaces
25342024-10-17T22:41:54.820ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = DELETE path = /v1/network-interfaces/{interface}
25352024-10-17T22:41:54.822ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/network-interfaces/{interface}
25362024-10-17T22:41:54.822ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = PUT path = /v1/network-interfaces/{interface}
25372024-10-17T22:41:54.822ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/ping
25382024-10-17T22:41:54.822ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/policy
25392024-10-17T22:41:54.822ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = PUT path = /v1/policy
25402024-10-17T22:41:54.822ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/projects
25412024-10-17T22:41:54.822ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/projects
25422024-10-17T22:41:54.822ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = DELETE path = /v1/projects/{project}
25432024-10-17T22:41:54.822ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/projects/{project}
25442024-10-17T22:41:54.822ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = PUT path = /v1/projects/{project}
25452024-10-17T22:41:54.822ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/projects/{project}/policy
25462024-10-17T22:41:54.822ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = PUT path = /v1/projects/{project}/policy
25472024-10-17T22:41:54.823ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/snapshots
25482024-10-17T22:41:54.823ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/snapshots
25492024-10-17T22:41:54.823ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = DELETE path = /v1/snapshots/{snapshot}
25502024-10-17T22:41:54.823ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/snapshots/{snapshot}
25512024-10-17T22:41:54.824ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/hardware/disks
25522024-10-17T22:41:54.824ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/hardware/disks/{disk_id}
25532024-10-17T22:41:54.824ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/hardware/racks
25542024-10-17T22:41:54.824ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/hardware/racks/{rack_id}
25552024-10-17T22:41:54.824ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/hardware/sleds
25562024-10-17T22:41:54.824ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/system/hardware/sleds
25572024-10-17T22:41:54.824ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/hardware/sleds/{sled_id}
25582024-10-17T22:41:54.824ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/hardware/sleds/{sled_id}/disks
25592024-10-17T22:41:54.824ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/hardware/sleds/{sled_id}/instances
25602024-10-17T22:41:54.824ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = PUT path = /v1/system/hardware/sleds/{sled_id}/provision-policy
25612024-10-17T22:41:54.824ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/hardware/sleds-uninitialized
25622024-10-17T22:41:54.825ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/hardware/switch-port
25632024-10-17T22:41:54.825ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = DELETE path = /v1/system/hardware/switch-port/{port}/settings
25642024-10-17T22:41:54.825ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/system/hardware/switch-port/{port}/settings
25652024-10-17T22:41:54.825ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/hardware/switch-port/{port}/status
25662024-10-17T22:41:54.825ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/hardware/switches
25672024-10-17T22:41:54.825ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/hardware/switches/{switch_id}
25682024-10-17T22:41:54.825ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/identity-providers
25692024-10-17T22:41:54.825ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/system/identity-providers/local/users
25702024-10-17T22:41:54.825ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = DELETE path = /v1/system/identity-providers/local/users/{user_id}
25712024-10-17T22:41:54.825ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/system/identity-providers/local/users/{user_id}/set-password
25722024-10-17T22:41:54.826ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/system/identity-providers/saml
25732024-10-17T22:41:54.826ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/identity-providers/saml/{provider}
25742024-10-17T22:41:54.826ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/ip-pools
25752024-10-17T22:41:54.826ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/system/ip-pools
25762024-10-17T22:41:54.826ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = DELETE path = /v1/system/ip-pools/{pool}
25772024-10-17T22:41:54.826ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/ip-pools/{pool}
25782024-10-17T22:41:54.826ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = PUT path = /v1/system/ip-pools/{pool}
25792024-10-17T22:41:54.826ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/ip-pools/{pool}/ranges
25802024-10-17T22:41:54.827ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/system/ip-pools/{pool}/ranges/add
25812024-10-17T22:41:54.827ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/system/ip-pools/{pool}/ranges/remove
25822024-10-17T22:41:54.827ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/ip-pools/{pool}/silos
25832024-10-17T22:41:54.827ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/system/ip-pools/{pool}/silos
25842024-10-17T22:41:54.827ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = DELETE path = /v1/system/ip-pools/{pool}/silos/{silo}
25852024-10-17T22:41:54.827ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = PUT path = /v1/system/ip-pools/{pool}/silos/{silo}
25862024-10-17T22:41:54.827ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/ip-pools/{pool}/utilization
25872024-10-17T22:41:54.827ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/ip-pools-service
25882024-10-17T22:41:54.827ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/ip-pools-service/ranges
25892024-10-17T22:41:54.827ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/system/ip-pools-service/ranges/add
25902024-10-17T22:41:54.827ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/system/ip-pools-service/ranges/remove
25912024-10-17T22:41:54.827ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/metrics/{metric_name}
25922024-10-17T22:41:54.827ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/networking/address-lot
25932024-10-17T22:41:54.828ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/system/networking/address-lot
25942024-10-17T22:41:54.828ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = DELETE path = /v1/system/networking/address-lot/{address_lot}
25952024-10-17T22:41:54.828ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/networking/address-lot/{address_lot}/blocks
25962024-10-17T22:41:54.828ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/networking/allow-list
25972024-10-17T22:41:54.828ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = PUT path = /v1/system/networking/allow-list
25982024-10-17T22:41:54.828ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/system/networking/bfd-disable
25992024-10-17T22:41:54.828ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/system/networking/bfd-enable
26002024-10-17T22:41:54.828ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/networking/bfd-status
26012024-10-17T22:41:54.828ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = DELETE path = /v1/system/networking/bgp
26022024-10-17T22:41:54.828ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/networking/bgp
26032024-10-17T22:41:54.828ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/system/networking/bgp
26042024-10-17T22:41:54.828ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/networking/bgp-announce-set
26052024-10-17T22:41:54.828ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = PUT path = /v1/system/networking/bgp-announce-set
26062024-10-17T22:41:54.828ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = DELETE path = /v1/system/networking/bgp-announce-set/{announce_set}
26072024-10-17T22:41:54.829ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/networking/bgp-announce-set/{announce_set}/announcement
26082024-10-17T22:41:54.829ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/networking/bgp-exported
26092024-10-17T22:41:54.829ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/networking/bgp-message-history
26102024-10-17T22:41:54.829ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/networking/bgp-routes-ipv4
26112024-10-17T22:41:54.829ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/networking/bgp-status
26122024-10-17T22:41:54.829ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/networking/loopback-address
26132024-10-17T22:41:54.829ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/system/networking/loopback-address
26142024-10-17T22:41:54.829ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = DELETE path = /v1/system/networking/loopback-address/{rack_id}/{switch_location}/{address}/{subnet_mask}
26152024-10-17T22:41:54.829ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = DELETE path = /v1/system/networking/switch-port-settings
26162024-10-17T22:41:54.829ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/networking/switch-port-settings
26172024-10-17T22:41:54.832ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/system/networking/switch-port-settings
26182024-10-17T22:41:54.832ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/networking/switch-port-settings/{port}
26192024-10-17T22:41:54.833ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/policy
26202024-10-17T22:41:54.834ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = PUT path = /v1/system/policy
26212024-10-17T22:41:54.834ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/roles
26222024-10-17T22:41:54.835ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/roles/{role_name}
26232024-10-17T22:41:54.835ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/silo-quotas
26242024-10-17T22:41:54.835ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/silos
26252024-10-17T22:41:54.835ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/system/silos
26262024-10-17T22:41:54.835ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = DELETE path = /v1/system/silos/{silo}
26272024-10-17T22:41:54.835ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/silos/{silo}
26282024-10-17T22:41:54.835ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/silos/{silo}/ip-pools
26292024-10-17T22:41:54.835ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/silos/{silo}/policy
26302024-10-17T22:41:54.835ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = PUT path = /v1/system/silos/{silo}/policy
26312024-10-17T22:41:54.835ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/silos/{silo}/quotas
26322024-10-17T22:41:54.835ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = PUT path = /v1/system/silos/{silo}/quotas
26332024-10-17T22:41:54.835ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = PUT path = /v1/system/update/repository
26342024-10-17T22:41:54.835ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/update/repository/{system_version}
26352024-10-17T22:41:54.836ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/users
26362024-10-17T22:41:54.836ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/users/{user_id}
26372024-10-17T22:41:54.836ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/users-builtin
26382024-10-17T22:41:54.837ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/users-builtin/{user}
26392024-10-17T22:41:54.837ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/utilization/silos
26402024-10-17T22:41:54.837ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/system/utilization/silos/{silo}
26412024-10-17T22:41:54.837ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/timeseries/query
26422024-10-17T22:41:54.837ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/timeseries/schema
26432024-10-17T22:41:54.838ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/users
26442024-10-17T22:41:54.838ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/utilization
26452024-10-17T22:41:54.838ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/vpc-firewall-rules
26462024-10-17T22:41:54.839ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = PUT path = /v1/vpc-firewall-rules
26472024-10-17T22:41:54.839ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/vpc-router-routes
26482024-10-17T22:41:54.839ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/vpc-router-routes
26492024-10-17T22:41:54.839ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = DELETE path = /v1/vpc-router-routes/{route}
26502024-10-17T22:41:54.839ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/vpc-router-routes/{route}
26512024-10-17T22:41:54.839ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = PUT path = /v1/vpc-router-routes/{route}
26522024-10-17T22:41:54.839ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/vpc-routers
26532024-10-17T22:41:54.839ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/vpc-routers
26542024-10-17T22:41:54.840ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = DELETE path = /v1/vpc-routers/{router}
26552024-10-17T22:41:54.840ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/vpc-routers/{router}
26562024-10-17T22:41:54.840ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = PUT path = /v1/vpc-routers/{router}
26572024-10-17T22:41:54.840ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/vpc-subnets
26582024-10-17T22:41:54.840ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/vpc-subnets
26592024-10-17T22:41:54.840ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = DELETE path = /v1/vpc-subnets/{subnet}
26602024-10-17T22:41:54.840ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/vpc-subnets/{subnet}
26612024-10-17T22:41:54.840ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = PUT path = /v1/vpc-subnets/{subnet}
26622024-10-17T22:41:54.840ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/vpc-subnets/{subnet}/network-interfaces
26632024-10-17T22:41:54.841ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/vpcs
26642024-10-17T22:41:54.841ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = POST path = /v1/vpcs
26652024-10-17T22:41:54.841ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = DELETE path = /v1/vpcs/{vpc}
26662024-10-17T22:41:54.841ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = GET path = /v1/vpcs/{vpc}
26672024-10-17T22:41:54.841ZDEBGtest_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39497 method = PUT path = /v1/vpcs/{vpc}
26682024-10-17T22:41:54.841ZINFOtest_metrics (dropshot_external): listening local_addr = 127.0.0.1:39497
26692024-10-17T22:41:54.841ZDEBGtest_metrics (dropshot_external): successfully registered DTrace USDT probes local_addr = 127.0.0.1:39497
26702024-10-17T22:41:54.855ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /
26712024-10-17T22:41:54.855ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /access
26722024-10-17T22:41:54.855ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /assets/{path}
26732024-10-17T22:41:54.855ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /device/auth
26742024-10-17T22:41:54.855ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /device/confirm
26752024-10-17T22:41:54.855ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /device/success
26762024-10-17T22:41:54.855ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /device/token
26772024-10-17T22:41:54.855ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /device/verify
26782024-10-17T22:41:54.855ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /experimental/v1/probes
26792024-10-17T22:41:54.855ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /experimental/v1/probes
26802024-10-17T22:41:54.855ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = DELETE path = /experimental/v1/probes/{probe}
26812024-10-17T22:41:54.855ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /experimental/v1/probes/{probe}
26822024-10-17T22:41:54.855ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /images
26832024-10-17T22:41:54.855ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /login
26842024-10-17T22:41:54.855ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /login/{silo_name}/local
26852024-10-17T22:41:54.855ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /login/{silo_name}/saml/{provider_name}
26862024-10-17T22:41:54.855ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /login/{silo_name}/saml/{provider_name}
26872024-10-17T22:41:54.858ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /login/{silo_name}/saml/{provider_name}/redirect
26882024-10-17T22:41:54.858ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /lookup/{path}
26892024-10-17T22:41:54.859ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /projects/{path}
26902024-10-17T22:41:54.859ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /projects-new
26912024-10-17T22:41:54.859ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /settings/{path}
26922024-10-17T22:41:54.859ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /system/{path}
26932024-10-17T22:41:54.859ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /utilization
26942024-10-17T22:41:54.859ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/certificates
26952024-10-17T22:41:54.860ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/certificates
26962024-10-17T22:41:54.860ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = DELETE path = /v1/certificates/{certificate}
26972024-10-17T22:41:54.860ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/certificates/{certificate}
26982024-10-17T22:41:54.860ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/disks
26992024-10-17T22:41:54.860ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/disks
27002024-10-17T22:41:54.860ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = DELETE path = /v1/disks/{disk}
27012024-10-17T22:41:54.860ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/disks/{disk}
27022024-10-17T22:41:54.860ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/disks/{disk}/bulk-write
27032024-10-17T22:41:54.860ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/disks/{disk}/bulk-write-start
27042024-10-17T22:41:54.860ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/disks/{disk}/bulk-write-stop
27052024-10-17T22:41:54.860ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/disks/{disk}/finalize
27062024-10-17T22:41:54.860ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/disks/{disk}/metrics/{metric}
27072024-10-17T22:41:54.860ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/floating-ips
27082024-10-17T22:41:54.861ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/floating-ips
27092024-10-17T22:41:54.861ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = DELETE path = /v1/floating-ips/{floating_ip}
27102024-10-17T22:41:54.862ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/floating-ips/{floating_ip}
27112024-10-17T22:41:54.862ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = PUT path = /v1/floating-ips/{floating_ip}
27122024-10-17T22:41:54.862ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/floating-ips/{floating_ip}/attach
27132024-10-17T22:41:54.862ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/floating-ips/{floating_ip}/detach
27142024-10-17T22:41:54.862ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/groups
27152024-10-17T22:41:54.862ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/groups/{group_id}
27162024-10-17T22:41:54.862ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/images
27172024-10-17T22:41:54.862ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/images
27182024-10-17T22:41:54.862ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = DELETE path = /v1/images/{image}
27192024-10-17T22:41:54.862ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/images/{image}
27202024-10-17T22:41:54.862ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/images/{image}/demote
27212024-10-17T22:41:54.862ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/images/{image}/promote
27222024-10-17T22:41:54.862ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/instances
27232024-10-17T22:41:54.862ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/instances
27242024-10-17T22:41:54.862ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = DELETE path = /v1/instances/{instance}
27252024-10-17T22:41:54.862ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/instances/{instance}
27262024-10-17T22:41:54.862ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = PUT path = /v1/instances/{instance}
27272024-10-17T22:41:54.872ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/instances/{instance}/disks
27282024-10-17T22:41:54.872ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/instances/{instance}/disks/attach
27292024-10-17T22:41:54.872ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/instances/{instance}/disks/detach
27302024-10-17T22:41:54.872ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/instances/{instance}/external-ips
27312024-10-17T22:41:54.872ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = DELETE path = /v1/instances/{instance}/external-ips/ephemeral
27322024-10-17T22:41:54.872ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/instances/{instance}/external-ips/ephemeral
27332024-10-17T22:41:54.872ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/instances/{instance}/reboot
27342024-10-17T22:41:54.872ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/instances/{instance}/serial-console
27352024-10-17T22:41:54.873ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/instances/{instance}/serial-console/stream
27362024-10-17T22:41:54.874ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/instances/{instance}/ssh-public-keys
27372024-10-17T22:41:54.874ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/instances/{instance}/start
27382024-10-17T22:41:54.874ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/instances/{instance}/stop
27392024-10-17T22:41:54.874ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/internet-gateway-ip-addresses
27402024-10-17T22:41:54.874ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/internet-gateway-ip-addresses
27412024-10-17T22:41:54.874ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = DELETE path = /v1/internet-gateway-ip-addresses/{address}
27422024-10-17T22:41:54.874ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/internet-gateway-ip-pools
27432024-10-17T22:41:54.875ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/internet-gateway-ip-pools
27442024-10-17T22:41:54.875ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = DELETE path = /v1/internet-gateway-ip-pools/{pool}
27452024-10-17T22:41:54.876ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/internet-gateways
27462024-10-17T22:41:54.877ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/internet-gateways
27472024-10-17T22:41:54.878ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = DELETE path = /v1/internet-gateways/{gateway}
27482024-10-17T22:41:54.878ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/internet-gateways/{gateway}
27492024-10-17T22:41:54.879ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/ip-pools
27502024-10-17T22:41:54.880ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/ip-pools/{pool}
27512024-10-17T22:41:54.880ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/login/{silo_name}/local
27522024-10-17T22:41:54.880ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/logout
27532024-10-17T22:41:54.880ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/me
27542024-10-17T22:41:54.880ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/me/groups
27552024-10-17T22:41:54.880ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/me/ssh-keys
27562024-10-17T22:41:54.880ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/me/ssh-keys
27572024-10-17T22:41:54.880ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = DELETE path = /v1/me/ssh-keys/{ssh_key}
27582024-10-17T22:41:54.880ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/me/ssh-keys/{ssh_key}
27592024-10-17T22:41:54.880ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/metrics/{metric_name}
27602024-10-17T22:41:54.881ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/network-interfaces
27612024-10-17T22:41:54.881ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/network-interfaces
27622024-10-17T22:41:54.881ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = DELETE path = /v1/network-interfaces/{interface}
27632024-10-17T22:41:54.881ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/network-interfaces/{interface}
27642024-10-17T22:41:54.881ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = PUT path = /v1/network-interfaces/{interface}
27652024-10-17T22:41:54.881ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/ping
27662024-10-17T22:41:54.881ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/policy
27672024-10-17T22:41:54.881ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = PUT path = /v1/policy
27682024-10-17T22:41:54.881ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/projects
27692024-10-17T22:41:54.881ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/projects
27702024-10-17T22:41:54.882ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = DELETE path = /v1/projects/{project}
27712024-10-17T22:41:54.882ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/projects/{project}
27722024-10-17T22:41:54.884ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = PUT path = /v1/projects/{project}
27732024-10-17T22:41:54.884ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/projects/{project}/policy
27742024-10-17T22:41:54.884ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = PUT path = /v1/projects/{project}/policy
27752024-10-17T22:41:54.884ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/snapshots
27762024-10-17T22:41:54.884ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/snapshots
27772024-10-17T22:41:54.884ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = DELETE path = /v1/snapshots/{snapshot}
27782024-10-17T22:41:54.884ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/snapshots/{snapshot}
27792024-10-17T22:41:54.884ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/hardware/disks
27802024-10-17T22:41:54.884ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/hardware/disks/{disk_id}
27812024-10-17T22:41:54.884ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/hardware/racks
27822024-10-17T22:41:54.885ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/hardware/racks/{rack_id}
27832024-10-17T22:41:54.895ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/hardware/sleds
27842024-10-17T22:41:54.895ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/system/hardware/sleds
27852024-10-17T22:41:54.895ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/hardware/sleds/{sled_id}
27862024-10-17T22:41:54.895ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/hardware/sleds/{sled_id}/disks
27872024-10-17T22:41:54.895ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/hardware/sleds/{sled_id}/instances
27882024-10-17T22:41:54.895ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = PUT path = /v1/system/hardware/sleds/{sled_id}/provision-policy
27892024-10-17T22:41:54.895ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/hardware/sleds-uninitialized
27902024-10-17T22:41:54.899ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/hardware/switch-port
27912024-10-17T22:41:54.914ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = DELETE path = /v1/system/hardware/switch-port/{port}/settings
27922024-10-17T22:41:54.915ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/system/hardware/switch-port/{port}/settings
27932024-10-17T22:41:54.916ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/hardware/switch-port/{port}/status
27942024-10-17T22:41:54.916ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/hardware/switches
27952024-10-17T22:41:54.916ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/hardware/switches/{switch_id}
27962024-10-17T22:41:54.916ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/identity-providers
27972024-10-17T22:41:54.916ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/system/identity-providers/local/users
27982024-10-17T22:41:54.916ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = DELETE path = /v1/system/identity-providers/local/users/{user_id}
27992024-10-17T22:41:54.916ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/system/identity-providers/local/users/{user_id}/set-password
28002024-10-17T22:41:54.916ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/system/identity-providers/saml
28012024-10-17T22:41:54.916ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/identity-providers/saml/{provider}
28022024-10-17T22:41:54.916ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/ip-pools
28032024-10-17T22:41:54.916ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/system/ip-pools
28042024-10-17T22:41:54.917ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = DELETE path = /v1/system/ip-pools/{pool}
28052024-10-17T22:41:54.918ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/ip-pools/{pool}
28062024-10-17T22:41:54.918ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = PUT path = /v1/system/ip-pools/{pool}
28072024-10-17T22:41:54.918ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/ip-pools/{pool}/ranges
28082024-10-17T22:41:54.919ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/system/ip-pools/{pool}/ranges/add
28092024-10-17T22:41:54.919ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/system/ip-pools/{pool}/ranges/remove
28102024-10-17T22:41:54.919ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/ip-pools/{pool}/silos
28112024-10-17T22:41:54.919ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/system/ip-pools/{pool}/silos
28122024-10-17T22:41:54.920ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = DELETE path = /v1/system/ip-pools/{pool}/silos/{silo}
28132024-10-17T22:41:54.920ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = PUT path = /v1/system/ip-pools/{pool}/silos/{silo}
28142024-10-17T22:41:54.921ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/ip-pools/{pool}/utilization
28152024-10-17T22:41:54.921ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/ip-pools-service
28162024-10-17T22:41:54.921ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/ip-pools-service/ranges
28172024-10-17T22:41:54.921ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/system/ip-pools-service/ranges/add
28182024-10-17T22:41:54.923ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/system/ip-pools-service/ranges/remove
28192024-10-17T22:41:54.923ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/metrics/{metric_name}
28202024-10-17T22:41:54.923ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/networking/address-lot
28212024-10-17T22:41:54.923ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/system/networking/address-lot
28222024-10-17T22:41:54.923ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = DELETE path = /v1/system/networking/address-lot/{address_lot}
28232024-10-17T22:41:54.923ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/networking/address-lot/{address_lot}/blocks
28242024-10-17T22:41:54.923ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/networking/allow-list
28252024-10-17T22:41:54.923ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = PUT path = /v1/system/networking/allow-list
28262024-10-17T22:41:54.923ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/system/networking/bfd-disable
28272024-10-17T22:41:54.923ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/system/networking/bfd-enable
28282024-10-17T22:41:54.923ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/networking/bfd-status
28292024-10-17T22:41:54.923ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = DELETE path = /v1/system/networking/bgp
28302024-10-17T22:41:54.923ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/networking/bgp
28312024-10-17T22:41:54.924ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/system/networking/bgp
28322024-10-17T22:41:54.924ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/networking/bgp-announce-set
28332024-10-17T22:41:54.924ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = PUT path = /v1/system/networking/bgp-announce-set
28342024-10-17T22:41:54.924ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = DELETE path = /v1/system/networking/bgp-announce-set/{announce_set}
28352024-10-17T22:41:54.925ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/networking/bgp-announce-set/{announce_set}/announcement
28362024-10-17T22:41:54.925ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/networking/bgp-exported
28372024-10-17T22:41:54.925ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/networking/bgp-message-history
28382024-10-17T22:41:54.925ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/networking/bgp-routes-ipv4
28392024-10-17T22:41:54.925ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/networking/bgp-status
28402024-10-17T22:41:54.925ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/networking/loopback-address
28412024-10-17T22:41:54.925ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/system/networking/loopback-address
28422024-10-17T22:41:54.925ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = DELETE path = /v1/system/networking/loopback-address/{rack_id}/{switch_location}/{address}/{subnet_mask}
28432024-10-17T22:41:54.925ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = DELETE path = /v1/system/networking/switch-port-settings
28442024-10-17T22:41:54.930ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/networking/switch-port-settings
28452024-10-17T22:41:54.930ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/system/networking/switch-port-settings
28462024-10-17T22:41:54.930ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/networking/switch-port-settings/{port}
28472024-10-17T22:41:54.930ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/policy
28482024-10-17T22:41:54.930ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = PUT path = /v1/system/policy
28492024-10-17T22:41:54.930ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/roles
28502024-10-17T22:41:54.930ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/roles/{role_name}
28512024-10-17T22:41:54.930ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/silo-quotas
28522024-10-17T22:41:54.930ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/silos
28532024-10-17T22:41:54.930ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/system/silos
28542024-10-17T22:41:54.930ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = DELETE path = /v1/system/silos/{silo}
28552024-10-17T22:41:54.930ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/silos/{silo}
28562024-10-17T22:41:54.930ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/silos/{silo}/ip-pools
28572024-10-17T22:41:54.931ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/silos/{silo}/policy
28582024-10-17T22:41:54.932ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = PUT path = /v1/system/silos/{silo}/policy
28592024-10-17T22:41:54.932ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/silos/{silo}/quotas
28602024-10-17T22:41:54.932ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = PUT path = /v1/system/silos/{silo}/quotas
28612024-10-17T22:41:54.932ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = PUT path = /v1/system/update/repository
28622024-10-17T22:41:54.932ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/update/repository/{system_version}
28632024-10-17T22:41:54.932ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/users
28642024-10-17T22:41:54.932ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/users/{user_id}
28652024-10-17T22:41:54.932ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/users-builtin
28662024-10-17T22:41:54.932ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/users-builtin/{user}
28672024-10-17T22:41:54.932ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/utilization/silos
28682024-10-17T22:41:54.932ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/system/utilization/silos/{silo}
28692024-10-17T22:41:54.932ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/timeseries/query
28702024-10-17T22:41:54.932ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/timeseries/schema
28712024-10-17T22:41:54.933ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/users
28722024-10-17T22:41:54.934ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/utilization
28732024-10-17T22:41:54.934ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/vpc-firewall-rules
28742024-10-17T22:41:54.939ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = PUT path = /v1/vpc-firewall-rules
28752024-10-17T22:41:54.939ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/vpc-router-routes
28762024-10-17T22:41:54.940ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/vpc-router-routes
28772024-10-17T22:41:54.940ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = DELETE path = /v1/vpc-router-routes/{route}
28782024-10-17T22:41:54.940ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/vpc-router-routes/{route}
28792024-10-17T22:41:54.940ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = PUT path = /v1/vpc-router-routes/{route}
28802024-10-17T22:41:54.940ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/vpc-routers
28812024-10-17T22:41:54.940ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/vpc-routers
28822024-10-17T22:41:54.940ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = DELETE path = /v1/vpc-routers/{router}
28832024-10-17T22:41:54.940ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/vpc-routers/{router}
28842024-10-17T22:41:54.940ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = PUT path = /v1/vpc-routers/{router}
28852024-10-17T22:41:54.940ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/vpc-subnets
28862024-10-17T22:41:54.940ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/vpc-subnets
28872024-10-17T22:41:54.940ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = DELETE path = /v1/vpc-subnets/{subnet}
28882024-10-17T22:41:54.940ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/vpc-subnets/{subnet}
28892024-10-17T22:41:54.940ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = PUT path = /v1/vpc-subnets/{subnet}
28902024-10-17T22:41:54.943ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/vpc-subnets/{subnet}/network-interfaces
28912024-10-17T22:41:54.951ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/vpcs
28922024-10-17T22:41:54.951ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = POST path = /v1/vpcs
28932024-10-17T22:41:54.951ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = DELETE path = /v1/vpcs/{vpc}
28942024-10-17T22:41:54.953ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = GET path = /v1/vpcs/{vpc}
28952024-10-17T22:41:54.954ZDEBGtest_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:43815 method = PUT path = /v1/vpcs/{vpc}
28962024-10-17T22:41:54.955ZINFOtest_metrics (dropshot_external_techport): listening local_addr = 127.0.0.1:43815
28972024-10-17T22:41:54.955ZDEBGtest_metrics (dropshot_external_techport): successfully registered DTrace USDT probes local_addr = 127.0.0.1:43815
28982024-10-17T22:41:54.955ZDEBGtest_metrics (nexus-producer-server): registered DTrace probes
28992024-10-17T22:41:54.955ZDEBGtest_metrics (dropshot): registered endpoint local_addr = 127.0.0.1:45067 method = GET path = /{producer_id}
29002024-10-17T22:41:54.955ZINFOtest_metrics (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:197 local_addr = 127.0.0.1:45067
29012024-10-17T22:41:54.955ZDEBGtest_metrics (dropshot): successfully registered DTrace USDT probes local_addr = 127.0.0.1:45067
29022024-10-17T22:41:54.956ZDEBGtest_metrics (nexus-producer-server): Requested any available port, Dropshot server has been bound to 127.0.0.1:45067
29032024-10-17T22:41:54.956ZDEBGtest_metrics (nexus-producer-server): Nexus IP provided explicitly, will use it addr = 127.0.0.1:33311
29042024-10-17T22:41:54.956ZDEBGtest_metrics (nexus-producer-server): starting producer registration task
29052024-10-17T22:41:54.957ZINFOtest_metrics (nexus-producer-server): starting oximeter metric producer server address = 127.0.0.1:45067 file = oximeter/producer/src/lib.rs:283 interval = 10s producer_id = 017ff6a2-3b72-4882-9b0b-97601068f99c
29062024-10-17T22:41:54.957ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
29072024-10-17T22:41:54.957ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
29082024-10-17T22:41:54.957ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
29092024-10-17T22:41:54.957ZTRACtest_metrics (dropshot): incoming request local_addr = [::1]:40953 method = GET remote_addr = [::1]:57532 req_id = e017148c-b974-4f85-b7f8-26cb1efcf034 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
29102024-10-17T22:41:54.957ZDEBGtest_metrics (producer-registration-task): registering / renewing oximeter producer lease with Nexus
29112024-10-17T22:41:54.957ZDEBGtest_metrics (producer-registration-task): using nexus address for registration addr = 127.0.0.1:33311
29122024-10-17T22:41:54.957ZDEBGtest_metrics (producer-registration-task): client request body = Some(Body) method = POST uri = http://127.0.0.1:33311/metrics/producers
29132024-10-17T22:41:54.958ZINFOtest_metrics (dropshot): request completed latency_us = 37303 local_addr = [::1]:40953 method = GET remote_addr = [::1]:57532 req_id = e017148c-b974-4f85-b7f8-26cb1efcf034 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
29142024-10-17T22:41:54.958ZINFOtest_metrics (dropshot_internal): accepted connection local_addr = 127.0.0.1:33311 remote_addr = 127.0.0.1:36384
29152024-10-17T22:41:54.958ZDEBGtest_metrics (BackgroundTasks): client response SledAgent = http://[::1]:40953 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:40953/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "e017148c-b974-4f85-b7f8-26cb1efcf034", "content-length": "6440", "date": "Thu, 17 Oct 2024 22:41:54 GMT"} })
29162024-10-17T22:41:54.958ZDEBGtest_metrics (BackgroundTasks): finished collection background_task = inventory_collection
29172024-10-17T22:41:54.958ZTRACtest_metrics (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
29182024-10-17T22:41:54.958ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
29192024-10-17T22:41:54.958ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
29202024-10-17T22:41:54.958ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
29212024-10-17T22:41:54.958ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
29222024-10-17T22:41:54.958ZTRACtest_metrics (dropshot_internal): incoming request local_addr = 127.0.0.1:33311 method = POST remote_addr = 127.0.0.1:36384 req_id = 271a11e5-a220-439c-a449-b153b7078ddf uri = /metrics/producers
29232024-10-17T22:41:54.959ZTRACtest_metrics (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:33311 method = POST remote_addr = 127.0.0.1:36384 req_id = 271a11e5-a220-439c-a449-b153b7078ddf resource = Database uri = /metrics/producers
29242024-10-17T22:41:54.964ZDEBGtest_metrics (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:33311 method = POST remote_addr = 127.0.0.1:36384 req_id = 271a11e5-a220-439c-a449-b153b7078ddf roles = RoleSet { roles: {} } uri = /metrics/producers
29252024-10-17T22:41:54.964ZDEBGtest_metrics (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:33311 method = POST remote_addr = 127.0.0.1:36384 req_id = 271a11e5-a220-439c-a449-b153b7078ddf resource = Database result = Ok(()) uri = /metrics/producers
29262024-10-17T22:41:54.964ZTRACtest_metrics (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Sled { parent: Fleet, key: 039be560-54cc-49e3-88df-1a29dadbf913, lookup_type: ById(039be560-54cc-49e3-88df-1a29dadbf913) }
29272024-10-17T22:41:54.964ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
29282024-10-17T22:41:54.964ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
29292024-10-17T22:41:54.964ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
29302024-10-17T22:41:54.964ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
29312024-10-17T22:41:54.964ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29322024-10-17T22:41:54.966ZDEBGtest_metrics (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
29332024-10-17T22:41:54.966ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
29342024-10-17T22:41:54.966ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
29352024-10-17T22:41:54.967ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
29362024-10-17T22:41:54.971ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29372024-10-17T22:41:54.973ZDEBGtest_metrics (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Sled { parent: Fleet, key: 039be560-54cc-49e3-88df-1a29dadbf913, lookup_type: ById(039be560-54cc-49e3-88df-1a29dadbf913) } result = Ok(())
29382024-10-17T22:41:54.973ZTRACtest_metrics (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Sled { parent: Fleet, key: 039be560-54cc-49e3-88df-1a29dadbf913, lookup_type: ById(039be560-54cc-49e3-88df-1a29dadbf913) }
29392024-10-17T22:41:54.973ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
29402024-10-17T22:41:54.973ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
29412024-10-17T22:41:54.974ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
29422024-10-17T22:41:54.974ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
29432024-10-17T22:41:54.974ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29442024-10-17T22:41:54.974ZDEBGtest_metrics (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Sled { parent: Fleet, key: 039be560-54cc-49e3-88df-1a29dadbf913, lookup_type: ById(039be560-54cc-49e3-88df-1a29dadbf913) } result = Ok(())
29452024-10-17T22:41:54.974ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
29462024-10-17T22:41:54.976ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
29472024-10-17T22:41:54.976ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
29482024-10-17T22:41:54.976ZDEBGtest_metrics (init_with_steps): Running step start_oximeter
29492024-10-17T22:41:54.976ZDEBGtest_metrics: Starting Oximeter
29502024-10-17T22:41:54.976ZDEBGtest_metrics (oximeter): registered DTrace probes
29512024-10-17T22:41:54.976ZINFOtest_metrics (oximeter): starting oximeter server file = oximeter/collector/src/lib.rs:252
29522024-10-17T22:41:54.976ZDEBGtest_metrics (oximeter): creating ClickHouse client
29532024-10-17T22:41:54.978ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = SELECT MAX(value) FROM oximeter.version;
29542024-10-17T22:41:55.300ZINFOtest_metrics (dropshot_internal): request completed error_message_external = Service Unavailable error_message_internal = no Oximeter instances available for assignment latency_us = 403839 local_addr = 127.0.0.1:33311 method = POST remote_addr = 127.0.0.1:36384 req_id = 271a11e5-a220-439c-a449-b153b7078ddf response_code = 503 uri = /metrics/producers
29552024-10-17T22:41:55.306ZDEBGtest_metrics (producer-registration-task): client response result = Ok(Response { url: "http://127.0.0.1:33311/metrics/producers", status: 503, headers: {"content-type": "application/json", "x-request-id": "271a11e5-a220-439c-a449-b153b7078ddf", "content-length": "133", "date": "Thu, 17 Oct 2024 22:41:54 GMT"} })
29562024-10-17T22:41:55.308ZWARNtest_metrics (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 349.009866ms error = "Error Response: status: 503 Service Unavailable; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"271a11e5-a220-439c-a449-b153b7078ddf\\", \\"content-length\\": \\"133\\", \\"date\\": \\"Thu, 17 Oct 2024 22:41:54 GMT\\"}; value: Error { error_code: Some(\\"ServiceNotAvailable\\"), message: \\"Service Unavailable\\", request_id: \\"271a11e5-a220-439c-a449-b153b7078ddf\\" }" file = oximeter/producer/src/lib.rs:424
29572024-10-17T22:41:55.337ZWARNtest_metrics (clickhouse-client): oximeter database does not exist, or is out-of-date collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/db/src/client/mod.rs:862 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
29582024-10-17T22:41:55.338ZDEBGtest_metrics (oximeter-agent): oximeter database does not exist, creating collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
29592024-10-17T22:41:55.338ZINFOtest_metrics (clickhouse-client): reading db version collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/db/src/client/mod.rs:800 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
29602024-10-17T22:41:55.338ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = SELECT MAX(value) FROM oximeter.version;
29612024-10-17T22:41:55.349ZWARNtest_metrics (clickhouse-client): oximeter database does not exist, or is out-of-date collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/db/src/client/mod.rs:862 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
29622024-10-17T22:41:55.349ZINFOtest_metrics (clickhouse-client): read oximeter database version collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/db/src/client/mod.rs:804 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 version = 0
29632024-10-17T22:41:55.349ZINFOtest_metrics (clickhouse-client): wiping and re-initializing oximeter schema collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/db/src/client/mod.rs:809 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
29642024-10-17T22:41:55.349ZDEBGtest_metrics (clickhouse-client): wiping ClickHouse database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
29652024-10-17T22:41:55.349ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = DROP DATABASE IF EXISTS oximeter
29662024-10-17T22:41:55.357ZDEBGtest_metrics (clickhouse-client): initializing ClickHouse database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
29672024-10-17T22:41:55.357ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = CREATE DATABASE IF NOT EXISTS oximeter
29682024-10-17T22:41:55.365ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = \n\n/* The version table contains metadata about the `oximeter` database */\nCREATE TABLE IF NOT EXISTS oximeter.version\n(\n value UInt64,\n timestamp DateTime64(9, 'UTC')\n)\nENGINE = MergeTree()\nORDER BY (value, timestamp)
29692024-10-17T22:41:55.383ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = \n\n/* The measurement tables contain all individual samples from each timeseries.\n *\n * Each table stores a single datum type, and otherwise contains nearly the same\n * structure. The primary sorting key is on the timeseries name, key, and then\n * timestamp, so that all timeseries from the same schema are grouped, followed\n * by all samples from the same timeseries.\n *\n * This reflects that one usually looks up the _key_ in one or more field table,\n * and then uses that to index quickly into the measurements tables.\n */\nCREATE TABLE IF NOT EXISTS oximeter.measurements_bool\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(UInt8)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29702024-10-17T22:41:55.404ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_i8\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Int8)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29712024-10-17T22:41:55.429ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_u8\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(UInt8)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29722024-10-17T22:41:55.471ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_i16\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Int16)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29732024-10-17T22:41:55.490ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_u16\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(UInt16)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29742024-10-17T22:41:55.507ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_i32\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Int32)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29752024-10-17T22:41:55.524ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_u32\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(UInt32)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29762024-10-17T22:41:55.545ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_i64\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Int64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29772024-10-17T22:41:55.576ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_u64\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(UInt64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29782024-10-17T22:41:55.595ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_f32\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Float32)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29792024-10-17T22:41:55.620ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_f64\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29802024-10-17T22:41:55.639ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_string\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(String)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29812024-10-17T22:41:55.656ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_bytes\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n /*\n * NOTE: Right now we can't unambiguously record a nullable byte array.\n * Arrays cannot be nested in `Nullable()` types, and encoding the array as\n * a string isn't palatable for a few reasons.\n * See: https://github.com/oxidecomputer/omicron/issues/4551 for more\n * details.\n */\n datum Array(UInt8)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29822024-10-17T22:41:55.698ZDEBGtest_metrics (producer-registration-task): client request body = Some(Body) method = POST uri = http://127.0.0.1:33311/metrics/producers
29832024-10-17T22:41:55.701ZINFOtest_metrics (dropshot_internal): accepted connection local_addr = 127.0.0.1:33311 remote_addr = 127.0.0.1:36386
29842024-10-17T22:41:55.701ZTRACtest_metrics (dropshot_internal): incoming request local_addr = 127.0.0.1:33311 method = POST remote_addr = 127.0.0.1:36386 req_id = 469afcfd-bb72-4d04-aff4-4a92101091ab uri = /metrics/producers
29852024-10-17T22:41:55.701ZTRACtest_metrics (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:33311 method = POST remote_addr = 127.0.0.1:36386 req_id = 469afcfd-bb72-4d04-aff4-4a92101091ab resource = Database uri = /metrics/producers
29862024-10-17T22:41:55.701ZDEBGtest_metrics (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:33311 method = POST remote_addr = 127.0.0.1:36386 req_id = 469afcfd-bb72-4d04-aff4-4a92101091ab roles = RoleSet { roles: {} } uri = /metrics/producers
29872024-10-17T22:41:55.701ZDEBGtest_metrics (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:33311 method = POST remote_addr = 127.0.0.1:36386 req_id = 469afcfd-bb72-4d04-aff4-4a92101091ab resource = Database result = Ok(()) uri = /metrics/producers
29882024-10-17T22:41:55.715ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativei64\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Int64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29892024-10-17T22:41:55.729ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativeu64\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(UInt64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29902024-10-17T22:41:55.754ZINFOtest_metrics (dropshot_internal): request completed error_message_external = Service Unavailable error_message_internal = no Oximeter instances available for assignment latency_us = 54026 local_addr = 127.0.0.1:33311 method = POST remote_addr = 127.0.0.1:36386 req_id = 469afcfd-bb72-4d04-aff4-4a92101091ab response_code = 503 uri = /metrics/producers
29912024-10-17T22:41:55.754ZDEBGtest_metrics (producer-registration-task): client response result = Ok(Response { url: "http://127.0.0.1:33311/metrics/producers", status: 503, headers: {"content-type": "application/json", "x-request-id": "469afcfd-bb72-4d04-aff4-4a92101091ab", "content-length": "133", "date": "Thu, 17 Oct 2024 22:41:54 GMT"} })
29922024-10-17T22:41:55.756ZWARNtest_metrics (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 376.537862ms error = "Error Response: status: 503 Service Unavailable; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"469afcfd-bb72-4d04-aff4-4a92101091ab\\", \\"content-length\\": \\"133\\", \\"date\\": \\"Thu, 17 Oct 2024 22:41:54 GMT\\"}; value: Error { error_code: Some(\\"ServiceNotAvailable\\"), message: \\"Service Unavailable\\", request_id: \\"469afcfd-bb72-4d04-aff4-4a92101091ab\\" }" file = oximeter/producer/src/lib.rs:424
29932024-10-17T22:41:55.758ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativef32\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Float32)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29942024-10-17T22:41:55.775ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = \n\n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativef64\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29952024-10-17T22:41:55.794ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogrami8\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n /*\n * NOTE: Array types cannot be Nullable, see\n * https://clickhouse.com/docs/en/sql-reference/data-types/nullable\n * for more details.\n *\n * This means we need to use empty arrays to indicate a missing value. This\n * is unfortunate, and at this point relies on the fact that an\n * `oximeter::Histogram` cannot have zero bins. If that changes, we'll need\n * to figure out another way to represent missing samples here.\n */\n bins Array(Int8),\n counts Array(UInt64),\n min Int8,\n max Int8,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29962024-10-17T22:41:55.810ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogramu8\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(UInt8),\n counts Array(UInt64),\n min UInt8,\n max UInt8,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29972024-10-17T22:41:55.831ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogrami16\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(Int16),\n counts Array(UInt64),\n min Int16,\n max Int16,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29982024-10-17T22:41:55.845ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogramu16\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(UInt16),\n counts Array(UInt64),\n min UInt16,\n max UInt16,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29992024-10-17T22:41:55.865ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogrami32\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(Int32),\n counts Array(UInt64),\n min Int32,\n max Int32,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30002024-10-17T22:41:55.880ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogramu32\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(UInt32),\n counts Array(UInt64),\n min UInt32,\n max UInt32,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30012024-10-17T22:41:55.896ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogrami64\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(Int64),\n counts Array(UInt64),\n min Int64,\n max Int64,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30022024-10-17T22:41:55.907ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogramu64\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(UInt64),\n counts Array(UInt64),\n min UInt64,\n max UInt64,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30032024-10-17T22:41:55.924ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogramf32\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(Float32),\n counts Array(UInt64),\n min Float32,\n max Float32,\n sum_of_samples Float64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30042024-10-17T22:41:55.945ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogramf64\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(Float64),\n counts Array(UInt64),\n min Float64,\n max Float64,\n sum_of_samples Float64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30052024-10-17T22:41:55.966ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = \n\n/* The field tables store named dimensions of each timeseries.\n *\n * As with the measurement tables, there is one field table for each field data\n * type. Fields are deduplicated by using the "replacing merge tree", though\n * this behavior **must not** be relied upon for query correctness.\n *\n * The index for the fields differs from the measurements, however. Rows are\n * sorted by timeseries name, then field name, field value, and finally\n * timeseries key. This reflects the most common pattern for looking them up:\n * by field name and possibly value, within a timeseries. The resulting keys are\n * usually then used to look up measurements.\n *\n * NOTE: We may want to consider a secondary index on these tables, sorting by\n * timeseries name and then key, since it would improve lookups where one\n * already has the key. Realistically though, these tables are quite small and\n * so performance benefits will be low in absolute terms.\n *\n * TTL: We use a materialized column to expire old field table records. This\n * column is generated automatically by the database whenever a new row is\n * inserted. It cannot be inserted directly, nor is it returned in a `SELECT *`\n * query. Since these tables are `ReplacingMergeTree`s, that means the last\n * record will remain during a deduplication, which will have the last\n * timestamp. ClickHouse will then expire old data for us, similar to the\n * measurement tables.\n */\nCREATE TABLE IF NOT EXISTS oximeter.fields_bool\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value UInt8,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30062024-10-17T22:41:55.981ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_i8\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value Int8,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30072024-10-17T22:41:55.997ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_u8\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value UInt8,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30082024-10-17T22:41:56.017ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_i16\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value Int16,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30092024-10-17T22:41:56.028ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_u16\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value UInt16,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30102024-10-17T22:41:56.040ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_i32\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value Int32,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30112024-10-17T22:41:56.077ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_u32\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value UInt32,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30122024-10-17T22:41:56.109ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_i64\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value Int64,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30132024-10-17T22:41:56.127ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_u64\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value UInt64,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30142024-10-17T22:41:56.173ZDEBGtest_metrics (producer-registration-task): client request body = Some(Body) method = POST uri = http://127.0.0.1:33311/metrics/producers
30152024-10-17T22:41:56.174ZINFOtest_metrics (dropshot_internal): accepted connection local_addr = 127.0.0.1:33311 remote_addr = 127.0.0.1:36388
30162024-10-17T22:41:56.175ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_ipaddr\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value IPv6,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30172024-10-17T22:41:56.175ZTRACtest_metrics (dropshot_internal): incoming request local_addr = 127.0.0.1:33311 method = POST remote_addr = 127.0.0.1:36388 req_id = 8f9ac626-43e3-4272-89e8-bc7da1b10110 uri = /metrics/producers
30182024-10-17T22:41:56.176ZTRACtest_metrics (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:33311 method = POST remote_addr = 127.0.0.1:36388 req_id = 8f9ac626-43e3-4272-89e8-bc7da1b10110 resource = Database uri = /metrics/producers
30192024-10-17T22:41:56.176ZDEBGtest_metrics (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:33311 method = POST remote_addr = 127.0.0.1:36388 req_id = 8f9ac626-43e3-4272-89e8-bc7da1b10110 roles = RoleSet { roles: {} } uri = /metrics/producers
30202024-10-17T22:41:56.177ZDEBGtest_metrics (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:33311 method = POST remote_addr = 127.0.0.1:36388 req_id = 8f9ac626-43e3-4272-89e8-bc7da1b10110 resource = Database result = Ok(()) uri = /metrics/producers
30212024-10-17T22:41:56.182ZINFOtest_metrics (dropshot_internal): request completed error_message_external = Service Unavailable error_message_internal = no Oximeter instances available for assignment latency_us = 6537 local_addr = 127.0.0.1:33311 method = POST remote_addr = 127.0.0.1:36388 req_id = 8f9ac626-43e3-4272-89e8-bc7da1b10110 response_code = 503 uri = /metrics/producers
30222024-10-17T22:41:56.184ZINFOtest_metrics (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 1b3228a8-106c-4cda-9e3a-742e41b95979
30232024-10-17T22:41:56.184ZDEBGtest_metrics (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 1b3228a8-106c-4cda-9e3a-742e41b95979 time_started = 2024-10-17 22:41:54.227581 UTC
30242024-10-17T22:41:56.184ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 2.295904289s iteration = 4 status = {"collection_id":"1b3228a8-106c-4cda-9e3a-742e41b95979","time_done":"2024-10-17 22:41:54.892990 UTC","time_started":"2024-10-17 22:41:54.227581 UTC"}
30252024-10-17T22:41:56.184ZDEBGtest_metrics (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 2 reason = Dependency
30262024-10-17T22:41:56.184ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 39.421\u{b5}s iteration = 2 status = {"error":"task disabled"}
30272024-10-17T22:41:56.184ZDEBGtest_metrics (producer-registration-task): client response result = Ok(Response { url: "http://127.0.0.1:33311/metrics/producers", status: 503, headers: {"content-type": "application/json", "x-request-id": "8f9ac626-43e3-4272-89e8-bc7da1b10110", "content-length": "133", "date": "Thu, 17 Oct 2024 22:41:56 GMT"} })
30282024-10-17T22:41:56.185ZWARNtest_metrics (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 694.75428ms error = "Error Response: status: 503 Service Unavailable; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"8f9ac626-43e3-4272-89e8-bc7da1b10110\\", \\"content-length\\": \\"133\\", \\"date\\": \\"Thu, 17 Oct 2024 22:41:56 GMT\\"}; value: Error { error_code: Some(\\"ServiceNotAvailable\\"), message: \\"Service Unavailable\\", request_id: \\"8f9ac626-43e3-4272-89e8-bc7da1b10110\\" }" file = oximeter/producer/src/lib.rs:424
30292024-10-17T22:41:56.189ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_string\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value String,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30302024-10-17T22:41:56.203ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_uuid\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value UUID,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30312024-10-17T22:41:56.217ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = \n\n/* The timeseries schema table stores the extracted schema for the samples\n * oximeter collects.\n */\nCREATE TABLE IF NOT EXISTS oximeter.timeseries_schema\n(\n timeseries_name String,\n fields Nested(\n name String,\n type Enum(\n 'Bool' = 1,\n 'I64' = 2,\n 'IpAddr' = 3,\n 'String' = 4,\n 'Uuid' = 6,\n 'I8' = 7,\n 'U8' = 8,\n 'I16' = 9,\n 'U16' = 10,\n 'I32' = 11,\n 'U32' = 12,\n 'U64' = 13\n ),\n source Enum(\n 'Target' = 1,\n 'Metric' = 2\n )\n ),\n datum_type Enum(\n 'Bool' = 1,\n 'I64' = 2,\n 'F64' = 3,\n 'String' = 4,\n 'Bytes' = 5,\n 'CumulativeI64' = 6,\n 'CumulativeF64' = 7,\n 'HistogramI64' = 8,\n 'HistogramF64' = 9,\n 'I8' = 10,\n 'U8' = 11,\n 'I16' = 12,\n 'U16' = 13,\n 'I32' = 14,\n 'U32' = 15,\n 'U64' = 16,\n 'F32' = 17,\n 'CumulativeU64' = 18,\n 'CumulativeF32' = 19,\n 'HistogramI8' = 20,\n 'HistogramU8' = 21,\n 'HistogramI16' = 22,\n 'HistogramU16' = 23,\n 'HistogramI32' = 24,\n 'HistogramU32' = 25,\n 'HistogramU64' = 26,\n 'HistogramF32' = 27\n ),\n created DateTime64(9, 'UTC')\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, fields.name)
30322024-10-17T22:41:56.230ZINFOtest_metrics (clickhouse-client): inserting current version collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/db/src/client/mod.rs:832 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 version = 12
30332024-10-17T22:41:56.230ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.version (*) VALUES (12, now());
30342024-10-17T22:41:56.241ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40755 method = GET path = /info
30352024-10-17T22:41:56.241ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40755 method = GET path = /producers
30362024-10-17T22:41:56.241ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40755 method = POST path = /producers
30372024-10-17T22:41:56.241ZDEBGtest_metrics (dropshot): registered endpoint local_addr = [::1]:40755 method = DELETE path = /producers/{producer_id}
30382024-10-17T22:41:56.242ZINFOtest_metrics (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:197 local_addr = [::1]:40755
30392024-10-17T22:41:56.243ZDEBGtest_metrics (dropshot): successfully registered DTrace USDT probes local_addr = [::1]:40755
30402024-10-17T22:41:56.243ZDEBGtest_metrics (oximeter): contacting nexus
30412024-10-17T22:41:56.601ZDEBGtest_metrics (oximeter): client request body = Some(Body) method = POST uri = http://127.0.0.1:33311/metrics/collectors
30422024-10-17T22:41:56.603ZINFOtest_metrics (dropshot_internal): accepted connection local_addr = 127.0.0.1:33311 remote_addr = 127.0.0.1:36390
30432024-10-17T22:41:56.604ZTRACtest_metrics (dns): buffer buffer = Length: 79 (0x4f) bytes\n0000: da d1 01 00 00 01 00 00 00 00 00 01 10 5f 63 72 ............._cr\n0010: 75 63 69 62 6c 65 2d 70 61 6e 74 72 79 04 5f 74 ucible-pantry._t\n0020: 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 cp.control-plane\n0030: 05 6f 78 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 .oxide.internal.\n0040: 00 21 00 01 00 00 29 04 d0 00 00 00 00 00 00 .!....)........ kind = dns peer_addr = [::1]:63256 req_id = c1e5f4e7-0dfd-497d-a76a-5c1a5ab91945
30442024-10-17T22:41:56.604ZDEBGtest_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 56017,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_crucible-pantry._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_crucible-pantry._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 16,\n 95,\n 99,\n 114,\n 117,\n 99,\n 105,\n 98,\n 108,\n 101,\n 45,\n 112,\n 97,\n 110,\n 116,\n 114,\n 121,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63256 req_id = c1e5f4e7-0dfd-497d-a76a-5c1a5ab91945
30452024-10-17T22:41:56.604ZDEBGtest_metrics (store): query key key = _crucible-pantry._tcp
30462024-10-17T22:41:56.604ZDEBGtest_metrics (store): query key key = 4cc306e2-a646-4794-a23e-b059c98afee3.host
30472024-10-17T22:41:56.604ZDEBGtest_metrics (dns): dns response additional_records = [Record { name_labels: Name("4cc306e2-a646-4794-a23e-b059c98afee3.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:63256 query = LowerQuery { name: LowerName(Name("_crucible-pantry._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_crucible-pantry._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_crucible-pantry._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 38587, target: Name("4cc306e2-a646-4794-a23e-b059c98afee3.host.control-plane.oxide.internal") })) }] req_id = c1e5f4e7-0dfd-497d-a76a-5c1a5ab91945
30482024-10-17T22:41:56.604ZTRACtest_metrics (dropshot_internal): incoming request local_addr = 127.0.0.1:33311 method = POST remote_addr = 127.0.0.1:36390 req_id = 7695b61e-c211-4e4f-ab5b-201fbf8c763b uri = /metrics/collectors
30492024-10-17T22:41:56.605ZTRACtest_metrics (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:33311 method = POST remote_addr = 127.0.0.1:36390 req_id = 7695b61e-c211-4e4f-ab5b-201fbf8c763b resource = Database uri = /metrics/collectors
30502024-10-17T22:41:56.606ZDEBGtest_metrics (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:33311 method = POST remote_addr = 127.0.0.1:36390 req_id = 7695b61e-c211-4e4f-ab5b-201fbf8c763b roles = RoleSet { roles: {} } uri = /metrics/collectors
30512024-10-17T22:41:56.606ZDEBGtest_metrics (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:33311 method = POST remote_addr = 127.0.0.1:36390 req_id = 7695b61e-c211-4e4f-ab5b-201fbf8c763b resource = Database result = Ok(()) uri = /metrics/collectors
30522024-10-17T22:41:56.606ZTRACtest_metrics (dns): buffer buffer = Length: 99 (0x63) bytes\n0000: 92 62 01 00 00 01 00 00 00 00 00 01 24 34 63 63 .b..........$4cc\n0010: 33 30 36 65 32 2d 61 36 34 36 2d 34 37 39 34 2d 306e2-a646-4794-\n0020: 61 32 33 65 2d 62 30 35 39 63 39 38 61 66 65 65 a23e-b059c98afee\n0030: 33 04 68 6f 73 74 0d 63 6f 6e 74 72 6f 6c 2d 70 3.host.control-p\n0040: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0050: 6e 61 6c 00 00 1c 00 01 00 00 29 04 d0 00 00 00 nal.......).....\n0060: 00 00 00 ... kind = dns peer_addr = [::1]:49342 req_id = 6115d6cd-b51a-46ff-8938-8c1de26621c5
30532024-10-17T22:41:56.606ZDEBGtest_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 37474,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("4cc306e2-a646-4794-a23e-b059c98afee3.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("4cc306e2-a646-4794-a23e-b059c98afee3.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 36,\n 52,\n 99,\n 99,\n 51,\n 48,\n 54,\n 101,\n 50,\n 45,\n 97,\n 54,\n 52,\n 54,\n 45,\n 52,\n 55,\n 57,\n 52,\n 45,\n 97,\n 50,\n 51,\n 101,\n 45,\n 98,\n 48,\n 53,\n 57,\n 99,\n 57,\n 56,\n 97,\n 102,\n 101,\n 101,\n 51,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:49342 req_id = 6115d6cd-b51a-46ff-8938-8c1de26621c5
30542024-10-17T22:41:56.606ZDEBGtest_metrics (store): query key key = 4cc306e2-a646-4794-a23e-b059c98afee3.host
30552024-10-17T22:41:56.606ZDEBGtest_metrics (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:49342 query = LowerQuery { name: LowerName(Name("4cc306e2-a646-4794-a23e-b059c98afee3.host.control-plane.oxide.internal.")), original: Query { name: Name("4cc306e2-a646-4794-a23e-b059c98afee3.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("4cc306e2-a646-4794-a23e-b059c98afee3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 6115d6cd-b51a-46ff-8938-8c1de26621c5
30562024-10-17T22:41:56.936ZDEBGtest_metrics (producer-registration-task): client request body = Some(Body) method = POST uri = http://127.0.0.1:33311/metrics/producers
30572024-10-17T22:41:56.936ZINFOtest_metrics (nexus): registered new oximeter metric collection server address = [::1]:40755 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78
30582024-10-17T22:41:56.936ZINFOtest_metrics (dropshot_internal): request completed latency_us = 332391 local_addr = 127.0.0.1:33311 method = POST remote_addr = 127.0.0.1:36390 req_id = 7695b61e-c211-4e4f-ab5b-201fbf8c763b response_code = 204 uri = /metrics/collectors
30592024-10-17T22:41:56.937ZINFOtest_metrics (dropshot_internal): accepted connection local_addr = 127.0.0.1:33311 remote_addr = 127.0.0.1:36392
30602024-10-17T22:41:56.937ZDEBGtest_metrics (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:33311/metrics/collectors", status: 204, headers: {"x-request-id": "7695b61e-c211-4e4f-ab5b-201fbf8c763b", "date": "Thu, 17 Oct 2024 22:41:56 GMT"} })
30612024-10-17T22:41:56.940ZINFOtest_metrics (oximeter): oximeter registered with nexus file = oximeter/collector/src/lib.rs:391 id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78
30622024-10-17T22:41:56.940ZDEBGtest_metrics (init_with_steps): Running step start_producer_server
30632024-10-17T22:41:56.940ZDEBGtest_metrics: Starting test metric Producer Server
30642024-10-17T22:41:56.940ZINFOtest_metrics (external client test context): client request body = Body { inner: BoxBody } method = POST uri = http://127.0.0.1:39497/v1/system/ip-pools
30652024-10-17T22:41:56.940ZTRACtest_metrics (dropshot_internal): incoming request local_addr = 127.0.0.1:33311 method = POST remote_addr = 127.0.0.1:36392 req_id = 64866010-7d32-49b7-8f5d-04b42831c4de uri = /metrics/producers
30662024-10-17T22:41:56.976ZTRACtest_metrics (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:33311 method = POST remote_addr = 127.0.0.1:36392 req_id = 64866010-7d32-49b7-8f5d-04b42831c4de resource = Database uri = /metrics/producers
30672024-10-17T22:41:56.978ZDEBGtest_metrics (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:33311 method = POST remote_addr = 127.0.0.1:36392 req_id = 64866010-7d32-49b7-8f5d-04b42831c4de roles = RoleSet { roles: {} } uri = /metrics/producers
30682024-10-17T22:41:56.978ZDEBGtest_metrics (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:33311 method = POST remote_addr = 127.0.0.1:36392 req_id = 64866010-7d32-49b7-8f5d-04b42831c4de resource = Database result = Ok(()) uri = /metrics/producers
30692024-10-17T22:41:56.978ZINFOtest_metrics (dropshot_external): accepted connection local_addr = 127.0.0.1:39497 remote_addr = 127.0.0.1:46292
30702024-10-17T22:41:56.979ZINFOtest_metrics (dropshot_internal): accepted connection local_addr = 127.0.0.1:33311 remote_addr = 127.0.0.1:36394
30712024-10-17T22:41:56.979ZINFOtest_metrics (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:761
30722024-10-17T22:41:56.979ZDEBGtest_metrics (oximeter): client request body = None method = GET uri = http://127.0.0.1:33311/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
30732024-10-17T22:41:56.981ZTRACtest_metrics (dropshot_internal): incoming request local_addr = 127.0.0.1:33311 method = POST remote_addr = 127.0.0.1:36394 req_id = 50b6c175-6698-4264-9145-a0c78bc972f8 uri = /metrics/producers
30742024-10-17T22:41:56.984ZTRACtest_metrics (dropshot_internal): incoming request local_addr = 127.0.0.1:33311 method = GET remote_addr = 127.0.0.1:36390 req_id = df32f77a-897f-465f-aae0-f67a46084e77 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
30752024-10-17T22:41:56.984ZTRACtest_metrics (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:33311 method = POST remote_addr = 127.0.0.1:36394 req_id = 50b6c175-6698-4264-9145-a0c78bc972f8 resource = Database uri = /metrics/producers
30762024-10-17T22:41:56.984ZDEBGtest_metrics (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:33311 method = POST remote_addr = 127.0.0.1:36394 req_id = 50b6c175-6698-4264-9145-a0c78bc972f8 roles = RoleSet { roles: {} } uri = /metrics/producers
30772024-10-17T22:41:56.984ZDEBGtest_metrics (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:33311 method = POST remote_addr = 127.0.0.1:36394 req_id = 50b6c175-6698-4264-9145-a0c78bc972f8 resource = Database result = Ok(()) uri = /metrics/producers
30782024-10-17T22:41:56.984ZTRACtest_metrics (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:33311 method = GET remote_addr = 127.0.0.1:36390 req_id = df32f77a-897f-465f-aae0-f67a46084e77 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
30792024-10-17T22:41:56.984ZDEBGtest_metrics (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:33311 method = GET remote_addr = 127.0.0.1:36390 req_id = df32f77a-897f-465f-aae0-f67a46084e77 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
30802024-10-17T22:41:56.984ZDEBGtest_metrics (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:33311 method = GET remote_addr = 127.0.0.1:36390 req_id = df32f77a-897f-465f-aae0-f67a46084e77 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
30812024-10-17T22:41:56.994ZTRACtest_metrics (dropshot_external): incoming request local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = ed850e28-5c27-4387-90ab-cb17c1ad5084 uri = /v1/system/ip-pools
30822024-10-17T22:41:57.002ZTRACtest_metrics (dropshot_external): authn: trying SchemeName("spoof") local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = ed850e28-5c27-4387-90ab-cb17c1ad5084 uri = /v1/system/ip-pools
30832024-10-17T22:41:57.002ZDEBGtest_metrics (dropshot_external): looking up silo for user local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = ed850e28-5c27-4387-90ab-cb17c1ad5084 silo_user_id = 001de000-05e4-4000-8000-000000004007 uri = /v1/system/ip-pools
30842024-10-17T22:41:57.002ZTRACtest_metrics (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
30852024-10-17T22:41:57.002ZDEBGtest_metrics (ExternalAuthn): roles roles = RoleSet { roles: {} }
30862024-10-17T22:41:57.002ZDEBGtest_metrics (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
30872024-10-17T22:41:57.002ZTRACtest_metrics (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
30882024-10-17T22:41:57.002ZDEBGtest_metrics (ExternalAuthn): roles roles = RoleSet { roles: {} }
30892024-10-17T22:41:57.002ZDEBGtest_metrics (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
30902024-10-17T22:41:57.042ZINFOtest_metrics (nexus): registered oximeter collector client id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78
30912024-10-17T22:41:57.042ZDEBGtest_metrics (nexus): client request body = Some(Body) method = POST oximeter-collector = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 uri = http://[::1]:40755/producers
30922024-10-17T22:41:57.042ZTRACtest_metrics (ExternalAuthn): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 001de000-05e4-4000-8000-000000004007, lookup_type: ById(001de000-05e4-4000-8000-000000004007) }
30932024-10-17T22:41:57.042ZTRACtest_metrics (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo
30942024-10-17T22:41:57.042ZTRACtest_metrics (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
30952024-10-17T22:41:57.042ZDEBGtest_metrics (ExternalAuthn): roles roles = RoleSet { roles: {} }
30962024-10-17T22:41:57.043ZDEBGtest_metrics (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
30972024-10-17T22:41:57.044ZINFOtest_metrics (dropshot_internal): request completed latency_us = 64332 local_addr = 127.0.0.1:33311 method = GET remote_addr = 127.0.0.1:36390 req_id = df32f77a-897f-465f-aae0-f67a46084e77 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
30982024-10-17T22:41:57.075ZINFOtest_metrics (nexus): registered oximeter collector client id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78
30992024-10-17T22:41:57.075ZDEBGtest_metrics (nexus): client request body = Some(Body) method = POST oximeter-collector = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 uri = http://[::1]:40755/producers
31002024-10-17T22:41:57.077ZINFOtest_metrics (dropshot): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [::1]:40755 remote_addr = [::1]:47376
31012024-10-17T22:41:57.077ZINFOtest_metrics (dropshot): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [::1]:40755 remote_addr = [::1]:47378
31022024-10-17T22:41:57.077ZDEBGtest_metrics (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:33311/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "df32f77a-897f-465f-aae0-f67a46084e77", "content-length": "281", "date": "Thu, 17 Oct 2024 22:41:56 GMT"} })
31032024-10-17T22:41:57.078ZDEBGtest_metrics (oximeter): client request body = None method = GET uri = http://127.0.0.1:33311/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
31042024-10-17T22:41:57.078ZTRACtest_metrics (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
31052024-10-17T22:41:57.079ZTRACtest_metrics (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
31062024-10-17T22:41:57.079ZDEBGtest_metrics (ExternalAuthn): roles roles = RoleSet { roles: {} }
31072024-10-17T22:41:57.080ZDEBGtest_metrics (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
31082024-10-17T22:41:57.082ZTRACtest_metrics (dropshot): incoming request local_addr = [::1]:40755 method = POST remote_addr = [::1]:47376 req_id = 2731ff4d-5d3f-4bde-83a5-985bb158da82 uri = /producers
31092024-10-17T22:41:57.082ZTRACtest_metrics (dropshot): incoming request local_addr = [::1]:40755 method = POST remote_addr = [::1]:47378 req_id = 92923bc3-1301-427c-b461-5b03f9c7029c uri = /producers
31102024-10-17T22:41:57.082ZTRACtest_metrics (dropshot_internal): incoming request local_addr = 127.0.0.1:33311 method = GET remote_addr = 127.0.0.1:36390 req_id = 497cb58e-9d4c-4e77-b74e-543d8c755069 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
31112024-10-17T22:41:57.082ZDEBGtest_metrics (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
31122024-10-17T22:41:57.086ZDEBGtest_metrics (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 001de000-05e4-4000-8000-000000004007, lookup_type: ById(001de000-05e4-4000-8000-000000004007) } result = Ok(())
31132024-10-17T22:41:57.086ZTRACtest_metrics (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
31142024-10-17T22:41:57.086ZDEBGtest_metrics (ExternalAuthn): roles roles = RoleSet { roles: {} }
31152024-10-17T22:41:57.086ZDEBGtest_metrics (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
31162024-10-17T22:41:57.088ZDEBGtest_metrics (oximeter-agent): registered new metric producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
31172024-10-17T22:41:57.088ZDEBGtest_metrics (oximeter-agent): registered new metric producer address = 127.0.0.1:45067 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 017ff6a2-3b72-4882-9b0b-97601068f99c
31182024-10-17T22:41:57.088ZTRACtest_metrics (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:33311 method = GET remote_addr = 127.0.0.1:36390 req_id = 497cb58e-9d4c-4e77-b74e-543d8c755069 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
31192024-10-17T22:41:57.090ZDEBGtest_metrics (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:33311 method = GET remote_addr = 127.0.0.1:36390 req_id = 497cb58e-9d4c-4e77-b74e-543d8c755069 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
31202024-10-17T22:41:57.090ZDEBGtest_metrics (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:33311 method = GET remote_addr = 127.0.0.1:36390 req_id = 497cb58e-9d4c-4e77-b74e-543d8c755069 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
31212024-10-17T22:41:57.090ZINFOtest_metrics (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 8630 local_addr = [::1]:40755 method = POST remote_addr = [::1]:47376 req_id = 2731ff4d-5d3f-4bde-83a5-985bb158da82 response_code = 204 uri = /producers
31222024-10-17T22:41:57.130ZINFOtest_metrics (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 48920 local_addr = [::1]:40755 method = POST remote_addr = [::1]:47378 req_id = 92923bc3-1301-427c-b461-5b03f9c7029c response_code = 204 uri = /producers
31232024-10-17T22:41:57.162ZDEBGtest_metrics (collection-task): starting oximeter collection task address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 interval = 1s producer_id = a6458b7d-87c3-4483-be96-854d814c20de
31242024-10-17T22:41:57.163ZDEBGtest_metrics (collection-task): starting oximeter collection task address = 127.0.0.1:45067 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 interval = 10s producer_id = 017ff6a2-3b72-4882-9b0b-97601068f99c
31252024-10-17T22:41:57.163ZDEBGtest_metrics (nexus): client response oximeter-collector = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 result = Ok(Response { url: "http://[::1]:40755/producers", status: 204, headers: {"x-request-id": "2731ff4d-5d3f-4bde-83a5-985bb158da82", "date": "Thu, 17 Oct 2024 22:41:56 GMT"} })
31262024-10-17T22:41:57.164ZINFOtest_metrics (nexus): assigned collector to new producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
31272024-10-17T22:41:57.164ZDEBGtest_metrics (nexus): client response oximeter-collector = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 result = Ok(Response { url: "http://[::1]:40755/producers", status: 204, headers: {"x-request-id": "92923bc3-1301-427c-b461-5b03f9c7029c", "date": "Thu, 17 Oct 2024 22:41:56 GMT"} })
31282024-10-17T22:41:57.164ZINFOtest_metrics (nexus): assigned collector to new producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 producer_id = 017ff6a2-3b72-4882-9b0b-97601068f99c
31292024-10-17T22:41:57.165ZINFOtest_metrics (dropshot_internal): request completed latency_us = 186445 local_addr = 127.0.0.1:33311 method = POST remote_addr = 127.0.0.1:36394 req_id = 50b6c175-6698-4264-9145-a0c78bc972f8 response_code = 201 uri = /metrics/producers
31302024-10-17T22:41:57.166ZINFOtest_metrics (dropshot_internal): request completed latency_us = 227368 local_addr = 127.0.0.1:33311 method = POST remote_addr = 127.0.0.1:36392 req_id = 64866010-7d32-49b7-8f5d-04b42831c4de response_code = 201 uri = /metrics/producers
31312024-10-17T22:41:57.171ZDEBGtest_metrics (producer-registration-task): client response result = Ok(Response { url: "http://127.0.0.1:33311/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "64866010-7d32-49b7-8f5d-04b42831c4de", "content-length": "41", "date": "Thu, 17 Oct 2024 22:41:56 GMT"} })
31322024-10-17T22:41:57.172ZDEBGtest_metrics (producer-registration-task): registered with nexus successfully
31332024-10-17T22:41:57.173ZDEBGtest_metrics (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
31342024-10-17T22:41:57.176ZTRACtest_metrics (ExternalAuthn): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }
31352024-10-17T22:41:57.176ZTRACtest_metrics (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo
31362024-10-17T22:41:57.176ZTRACtest_metrics (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
31372024-10-17T22:41:57.176ZDEBGtest_metrics (ExternalAuthn): roles roles = RoleSet { roles: {} }
31382024-10-17T22:41:57.177ZDEBGtest_metrics (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
31392024-10-17T22:41:57.183ZINFOtest_metrics (dropshot_internal): request completed latency_us = 101662 local_addr = 127.0.0.1:33311 method = GET remote_addr = 127.0.0.1:36390 req_id = 497cb58e-9d4c-4e77-b74e-543d8c755069 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
31402024-10-17T22:41:57.184ZDEBGtest_metrics (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:33311/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "497cb58e-9d4c-4e77-b74e-543d8c755069", "content-length": "29", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
31412024-10-17T22:41:57.184ZDEBGtest_metrics (oximeter-agent): removed collection task from set collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 017ff6a2-3b72-4882-9b0b-97601068f99c
31422024-10-17T22:41:57.185ZDEBGtest_metrics (oximeter-agent): shut down collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 017ff6a2-3b72-4882-9b0b-97601068f99c
31432024-10-17T22:41:57.185ZDEBGtest_metrics (oximeter-agent): received request to register existing metric producer, updating collection information address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 interval = 1s producer_id = a6458b7d-87c3-4483-be96-854d814c20de
31442024-10-17T22:41:57.185ZINFOtest_metrics (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:809 n_current_tasks = 1 n_pruned_tasks = 1
31452024-10-17T22:41:57.185ZDEBGtest_metrics (collection-task): collection task received shutdown request address = 127.0.0.1:45067 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 017ff6a2-3b72-4882-9b0b-97601068f99c
31462024-10-17T22:41:57.185ZDEBGtest_metrics (collection-task): collection task received request to update its producer information address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 interval = 1s producer_id = a6458b7d-87c3-4483-be96-854d814c20de
31472024-10-17T22:41:57.186ZTRACtest_metrics (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
31482024-10-17T22:41:57.186ZTRACtest_metrics (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
31492024-10-17T22:41:57.186ZDEBGtest_metrics (ExternalAuthn): roles roles = RoleSet { roles: {} }
31502024-10-17T22:41:57.187ZDEBGtest_metrics (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
31512024-10-17T22:41:57.192ZDEBGtest_metrics (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
31522024-10-17T22:41:57.196ZDEBGtest_metrics (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) } result = Ok(())
31532024-10-17T22:41:57.198ZTRACtest_metrics (dropshot_external): authn result: Ok(Context { kind: Authenticated(Details { actor: Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } }, Some(SiloAuthnPolicy { mapped_fleet_roles: {} })), schemes_tried: [SchemeName("spoof")] }) local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = ed850e28-5c27-4387-90ab-cb17c1ad5084 uri = /v1/system/ip-pools
31542024-10-17T22:41:57.198ZTRACtest_metrics (dropshot_external): authorize begin action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = ed850e28-5c27-4387-90ab-cb17c1ad5084 resource = IpPoolList uri = /v1/system/ip-pools
31552024-10-17T22:41:57.198ZTRACtest_metrics (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = ed850e28-5c27-4387-90ab-cb17c1ad5084 resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet uri = /v1/system/ip-pools
31562024-10-17T22:41:57.198ZTRACtest_metrics (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = ed850e28-5c27-4387-90ab-cb17c1ad5084 resource = Database uri = /v1/system/ip-pools
31572024-10-17T22:41:57.198ZDEBGtest_metrics (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = ed850e28-5c27-4387-90ab-cb17c1ad5084 roles = RoleSet { roles: {} } uri = /v1/system/ip-pools
31582024-10-17T22:41:57.198ZDEBGtest_metrics (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = ed850e28-5c27-4387-90ab-cb17c1ad5084 resource = Database result = Ok(()) uri = /v1/system/ip-pools
31592024-10-17T22:41:57.213ZDEBGtest_metrics (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = ed850e28-5c27-4387-90ab-cb17c1ad5084 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /v1/system/ip-pools
31602024-10-17T22:41:57.217ZDEBGtest_metrics (dropshot_external): authorize result action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = ed850e28-5c27-4387-90ab-cb17c1ad5084 resource = IpPoolList result = Ok(()) uri = /v1/system/ip-pools
31612024-10-17T22:41:57.217ZTRACtest_metrics (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = ed850e28-5c27-4387-90ab-cb17c1ad5084 resource = Database uri = /v1/system/ip-pools
31622024-10-17T22:41:57.217ZDEBGtest_metrics (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = ed850e28-5c27-4387-90ab-cb17c1ad5084 roles = RoleSet { roles: {} } uri = /v1/system/ip-pools
31632024-10-17T22:41:57.218ZDEBGtest_metrics (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = ed850e28-5c27-4387-90ab-cb17c1ad5084 resource = Database result = Ok(()) uri = /v1/system/ip-pools
31642024-10-17T22:41:57.223ZINFOtest_metrics (dropshot_external): request completed latency_us = 241516 local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = ed850e28-5c27-4387-90ab-cb17c1ad5084 response_code = 201 uri = /v1/system/ip-pools
31652024-10-17T22:41:57.223ZINFOtest_metrics (external client test context): client received response status = 201
31662024-10-17T22:41:57.224ZINFOtest_metrics (external client test context): client request body = Body { inner: BoxBody } method = POST uri = http://127.0.0.1:39497/v1/system/ip-pools/default/ranges/add
31672024-10-17T22:41:57.224ZTRACtest_metrics (dropshot_external): incoming request local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = cfd7d47d-c815-4abc-9322-bd9cd799a583 uri = /v1/system/ip-pools/default/ranges/add
31682024-10-17T22:41:57.224ZTRACtest_metrics (dropshot_external): authn: trying SchemeName("spoof") local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = cfd7d47d-c815-4abc-9322-bd9cd799a583 uri = /v1/system/ip-pools/default/ranges/add
31692024-10-17T22:41:57.225ZDEBGtest_metrics (dropshot_external): looking up silo for user local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = cfd7d47d-c815-4abc-9322-bd9cd799a583 silo_user_id = 001de000-05e4-4000-8000-000000004007 uri = /v1/system/ip-pools/default/ranges/add
31702024-10-17T22:41:57.225ZTRACtest_metrics (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
31712024-10-17T22:41:57.225ZDEBGtest_metrics (ExternalAuthn): roles roles = RoleSet { roles: {} }
31722024-10-17T22:41:57.225ZDEBGtest_metrics (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
31732024-10-17T22:41:57.227ZTRACtest_metrics (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
31742024-10-17T22:41:57.227ZDEBGtest_metrics (ExternalAuthn): roles roles = RoleSet { roles: {} }
31752024-10-17T22:41:57.227ZDEBGtest_metrics (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
31762024-10-17T22:41:57.229ZTRACtest_metrics (ExternalAuthn): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 001de000-05e4-4000-8000-000000004007, lookup_type: ById(001de000-05e4-4000-8000-000000004007) }
31772024-10-17T22:41:57.229ZTRACtest_metrics (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo
31782024-10-17T22:41:57.229ZTRACtest_metrics (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
31792024-10-17T22:41:57.229ZDEBGtest_metrics (ExternalAuthn): roles roles = RoleSet { roles: {} }
31802024-10-17T22:41:57.230ZDEBGtest_metrics (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
31812024-10-17T22:41:57.231ZTRACtest_metrics (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
31822024-10-17T22:41:57.232ZTRACtest_metrics (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
31832024-10-17T22:41:57.232ZDEBGtest_metrics (ExternalAuthn): roles roles = RoleSet { roles: {} }
31842024-10-17T22:41:57.232ZDEBGtest_metrics (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
31852024-10-17T22:41:57.233ZDEBGtest_metrics (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
31862024-10-17T22:41:57.239ZDEBGtest_metrics (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 001de000-05e4-4000-8000-000000004007, lookup_type: ById(001de000-05e4-4000-8000-000000004007) } result = Ok(())
31872024-10-17T22:41:57.239ZTRACtest_metrics (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
31882024-10-17T22:41:57.239ZDEBGtest_metrics (ExternalAuthn): roles roles = RoleSet { roles: {} }
31892024-10-17T22:41:57.239ZDEBGtest_metrics (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
31902024-10-17T22:41:57.240ZTRACtest_metrics (ExternalAuthn): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }
31912024-10-17T22:41:57.241ZTRACtest_metrics (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo
31922024-10-17T22:41:57.241ZTRACtest_metrics (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
31932024-10-17T22:41:57.241ZDEBGtest_metrics (ExternalAuthn): roles roles = RoleSet { roles: {} }
31942024-10-17T22:41:57.241ZDEBGtest_metrics (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
31952024-10-17T22:41:57.243ZTRACtest_metrics (results-sink): batch interval expired, but no samples to insert collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
31962024-10-17T22:41:57.244ZTRACtest_metrics (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
31972024-10-17T22:41:57.244ZTRACtest_metrics (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
31982024-10-17T22:41:57.244ZDEBGtest_metrics (ExternalAuthn): roles roles = RoleSet { roles: {} }
31992024-10-17T22:41:57.244ZDEBGtest_metrics (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
32002024-10-17T22:41:57.254ZDEBGtest_metrics (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
32012024-10-17T22:41:57.257ZDEBGtest_metrics (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) } result = Ok(())
32022024-10-17T22:41:57.257ZTRACtest_metrics (dropshot_external): authn result: Ok(Context { kind: Authenticated(Details { actor: Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } }, Some(SiloAuthnPolicy { mapped_fleet_roles: {} })), schemes_tried: [SchemeName("spoof")] }) local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = cfd7d47d-c815-4abc-9322-bd9cd799a583 uri = /v1/system/ip-pools/default/ranges/add
32032024-10-17T22:41:57.257ZTRACtest_metrics (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = cfd7d47d-c815-4abc-9322-bd9cd799a583 resource = Database uri = /v1/system/ip-pools/default/ranges/add
32042024-10-17T22:41:57.258ZDEBGtest_metrics (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = cfd7d47d-c815-4abc-9322-bd9cd799a583 roles = RoleSet { roles: {} } uri = /v1/system/ip-pools/default/ranges/add
32052024-10-17T22:41:57.258ZDEBGtest_metrics (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = cfd7d47d-c815-4abc-9322-bd9cd799a583 resource = Database result = Ok(()) uri = /v1/system/ip-pools/default/ranges/add
32062024-10-17T22:41:57.260ZTRACtest_metrics (dropshot_external): authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = cfd7d47d-c815-4abc-9322-bd9cd799a583 resource = IpPool { parent: Fleet, key: ebef2ccd-1e4f-4648-941f-f4a7a5eb5e31, lookup_type: ByName("default") } uri = /v1/system/ip-pools/default/ranges/add
32072024-10-17T22:41:57.262ZTRACtest_metrics (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = cfd7d47d-c815-4abc-9322-bd9cd799a583 resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet uri = /v1/system/ip-pools/default/ranges/add
32082024-10-17T22:41:57.262ZTRACtest_metrics (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = cfd7d47d-c815-4abc-9322-bd9cd799a583 resource = Database uri = /v1/system/ip-pools/default/ranges/add
32092024-10-17T22:41:57.262ZDEBGtest_metrics (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = cfd7d47d-c815-4abc-9322-bd9cd799a583 roles = RoleSet { roles: {} } uri = /v1/system/ip-pools/default/ranges/add
32102024-10-17T22:41:57.262ZDEBGtest_metrics (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = cfd7d47d-c815-4abc-9322-bd9cd799a583 resource = Database result = Ok(()) uri = /v1/system/ip-pools/default/ranges/add
32112024-10-17T22:41:57.268ZDEBGtest_metrics (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = cfd7d47d-c815-4abc-9322-bd9cd799a583 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /v1/system/ip-pools/default/ranges/add
32122024-10-17T22:41:57.272ZDEBGtest_metrics (dropshot_external): authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = cfd7d47d-c815-4abc-9322-bd9cd799a583 resource = IpPool { parent: Fleet, key: ebef2ccd-1e4f-4648-941f-f4a7a5eb5e31, lookup_type: ByName("default") } result = Ok(()) uri = /v1/system/ip-pools/default/ranges/add
32132024-10-17T22:41:57.273ZTRACtest_metrics (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = cfd7d47d-c815-4abc-9322-bd9cd799a583 resource = Database uri = /v1/system/ip-pools/default/ranges/add
32142024-10-17T22:41:57.273ZDEBGtest_metrics (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = cfd7d47d-c815-4abc-9322-bd9cd799a583 roles = RoleSet { roles: {} } uri = /v1/system/ip-pools/default/ranges/add
32152024-10-17T22:41:57.273ZDEBGtest_metrics (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = cfd7d47d-c815-4abc-9322-bd9cd799a583 resource = Database result = Ok(()) uri = /v1/system/ip-pools/default/ranges/add
32162024-10-17T22:41:57.275ZTRACtest_metrics (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = cfd7d47d-c815-4abc-9322-bd9cd799a583 resource = Database uri = /v1/system/ip-pools/default/ranges/add
32172024-10-17T22:41:57.275ZDEBGtest_metrics (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = cfd7d47d-c815-4abc-9322-bd9cd799a583 roles = RoleSet { roles: {} } uri = /v1/system/ip-pools/default/ranges/add
32182024-10-17T22:41:57.276ZDEBGtest_metrics (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = cfd7d47d-c815-4abc-9322-bd9cd799a583 resource = Database result = Ok(()) uri = /v1/system/ip-pools/default/ranges/add
32192024-10-17T22:41:57.277ZTRACtest_metrics (dropshot_external): authorize begin action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = cfd7d47d-c815-4abc-9322-bd9cd799a583 resource = IpPool { parent: Fleet, key: ebef2ccd-1e4f-4648-941f-f4a7a5eb5e31, lookup_type: ByName("default") } uri = /v1/system/ip-pools/default/ranges/add
32202024-10-17T22:41:57.277ZTRACtest_metrics (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = cfd7d47d-c815-4abc-9322-bd9cd799a583 resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet uri = /v1/system/ip-pools/default/ranges/add
32212024-10-17T22:41:57.277ZTRACtest_metrics (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = cfd7d47d-c815-4abc-9322-bd9cd799a583 resource = Database uri = /v1/system/ip-pools/default/ranges/add
32222024-10-17T22:41:57.277ZDEBGtest_metrics (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = cfd7d47d-c815-4abc-9322-bd9cd799a583 roles = RoleSet { roles: {} } uri = /v1/system/ip-pools/default/ranges/add
32232024-10-17T22:41:57.277ZDEBGtest_metrics (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = cfd7d47d-c815-4abc-9322-bd9cd799a583 resource = Database result = Ok(()) uri = /v1/system/ip-pools/default/ranges/add
32242024-10-17T22:41:57.282ZDEBGtest_metrics (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = cfd7d47d-c815-4abc-9322-bd9cd799a583 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /v1/system/ip-pools/default/ranges/add
32252024-10-17T22:41:57.285ZDEBGtest_metrics (dropshot_external): authorize result action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = cfd7d47d-c815-4abc-9322-bd9cd799a583 resource = IpPool { parent: Fleet, key: ebef2ccd-1e4f-4648-941f-f4a7a5eb5e31, lookup_type: ByName("default") } result = Ok(()) uri = /v1/system/ip-pools/default/ranges/add
32262024-10-17T22:41:57.310ZINFOtest_metrics (dropshot_external): request completed latency_us = 85877 local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = cfd7d47d-c815-4abc-9322-bd9cd799a583 response_code = 201 uri = /v1/system/ip-pools/default/ranges/add
32272024-10-17T22:41:57.311ZINFOtest_metrics (external client test context): client received response status = 201
32282024-10-17T22:41:57.311ZINFOtest_metrics (external client test context): client request body = Body { inner: BoxBody } method = POST uri = http://127.0.0.1:39497/v1/system/ip-pools/default/silos
32292024-10-17T22:41:57.312ZTRACtest_metrics (dropshot_external): incoming request local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = f9240b89-53d4-4fad-8e28-5c7b52787427 uri = /v1/system/ip-pools/default/silos
32302024-10-17T22:41:57.312ZTRACtest_metrics (dropshot_external): authn: trying SchemeName("spoof") local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = f9240b89-53d4-4fad-8e28-5c7b52787427 uri = /v1/system/ip-pools/default/silos
32312024-10-17T22:41:57.313ZDEBGtest_metrics (dropshot_external): looking up silo for user local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = f9240b89-53d4-4fad-8e28-5c7b52787427 silo_user_id = 001de000-05e4-4000-8000-000000004007 uri = /v1/system/ip-pools/default/silos
32322024-10-17T22:41:57.313ZTRACtest_metrics (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
32332024-10-17T22:41:57.313ZDEBGtest_metrics (ExternalAuthn): roles roles = RoleSet { roles: {} }
32342024-10-17T22:41:57.315ZDEBGtest_metrics (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
32352024-10-17T22:41:57.318ZTRACtest_metrics (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
32362024-10-17T22:41:57.320ZDEBGtest_metrics (ExternalAuthn): roles roles = RoleSet { roles: {} }
32372024-10-17T22:41:57.320ZDEBGtest_metrics (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
32382024-10-17T22:41:57.325ZTRACtest_metrics (ExternalAuthn): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 001de000-05e4-4000-8000-000000004007, lookup_type: ById(001de000-05e4-4000-8000-000000004007) }
32392024-10-17T22:41:57.325ZTRACtest_metrics (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo
32402024-10-17T22:41:57.325ZTRACtest_metrics (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
32412024-10-17T22:41:57.325ZDEBGtest_metrics (ExternalAuthn): roles roles = RoleSet { roles: {} }
32422024-10-17T22:41:57.332ZDEBGtest_metrics (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
32432024-10-17T22:41:57.334ZTRACtest_metrics (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
32442024-10-17T22:41:57.334ZTRACtest_metrics (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
32452024-10-17T22:41:57.334ZDEBGtest_metrics (ExternalAuthn): roles roles = RoleSet { roles: {} }
32462024-10-17T22:41:57.335ZDEBGtest_metrics (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
32472024-10-17T22:41:57.337ZDEBGtest_metrics (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
32482024-10-17T22:41:57.341ZDEBGtest_metrics (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 001de000-05e4-4000-8000-000000004007, lookup_type: ById(001de000-05e4-4000-8000-000000004007) } result = Ok(())
32492024-10-17T22:41:57.341ZTRACtest_metrics (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
32502024-10-17T22:41:57.341ZDEBGtest_metrics (ExternalAuthn): roles roles = RoleSet { roles: {} }
32512024-10-17T22:41:57.342ZDEBGtest_metrics (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
32522024-10-17T22:41:57.344ZTRACtest_metrics (ExternalAuthn): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }
32532024-10-17T22:41:57.344ZTRACtest_metrics (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo
32542024-10-17T22:41:57.344ZTRACtest_metrics (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
32552024-10-17T22:41:57.344ZDEBGtest_metrics (ExternalAuthn): roles roles = RoleSet { roles: {} }
32562024-10-17T22:41:57.345ZDEBGtest_metrics (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
32572024-10-17T22:41:57.347ZTRACtest_metrics (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
32582024-10-17T22:41:57.347ZTRACtest_metrics (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
32592024-10-17T22:41:57.347ZDEBGtest_metrics (ExternalAuthn): roles roles = RoleSet { roles: {} }
32602024-10-17T22:41:57.347ZDEBGtest_metrics (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
32612024-10-17T22:41:57.351ZDEBGtest_metrics (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
32622024-10-17T22:41:57.354ZDEBGtest_metrics (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) } result = Ok(())
32632024-10-17T22:41:57.354ZTRACtest_metrics (dropshot_external): authn result: Ok(Context { kind: Authenticated(Details { actor: Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } }, Some(SiloAuthnPolicy { mapped_fleet_roles: {} })), schemes_tried: [SchemeName("spoof")] }) local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = f9240b89-53d4-4fad-8e28-5c7b52787427 uri = /v1/system/ip-pools/default/silos
32642024-10-17T22:41:57.354ZTRACtest_metrics (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = f9240b89-53d4-4fad-8e28-5c7b52787427 resource = Database uri = /v1/system/ip-pools/default/silos
32652024-10-17T22:41:57.355ZDEBGtest_metrics (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = f9240b89-53d4-4fad-8e28-5c7b52787427 roles = RoleSet { roles: {} } uri = /v1/system/ip-pools/default/silos
32662024-10-17T22:41:57.355ZDEBGtest_metrics (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = f9240b89-53d4-4fad-8e28-5c7b52787427 resource = Database result = Ok(()) uri = /v1/system/ip-pools/default/silos
32672024-10-17T22:41:57.357ZTRACtest_metrics (dropshot_external): authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = f9240b89-53d4-4fad-8e28-5c7b52787427 resource = IpPool { parent: Fleet, key: ebef2ccd-1e4f-4648-941f-f4a7a5eb5e31, lookup_type: ByName("default") } uri = /v1/system/ip-pools/default/silos
32682024-10-17T22:41:57.358ZTRACtest_metrics (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = f9240b89-53d4-4fad-8e28-5c7b52787427 resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet uri = /v1/system/ip-pools/default/silos
32692024-10-17T22:41:57.358ZTRACtest_metrics (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = f9240b89-53d4-4fad-8e28-5c7b52787427 resource = Database uri = /v1/system/ip-pools/default/silos
32702024-10-17T22:41:57.358ZDEBGtest_metrics (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = f9240b89-53d4-4fad-8e28-5c7b52787427 roles = RoleSet { roles: {} } uri = /v1/system/ip-pools/default/silos
32712024-10-17T22:41:57.358ZDEBGtest_metrics (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = f9240b89-53d4-4fad-8e28-5c7b52787427 resource = Database result = Ok(()) uri = /v1/system/ip-pools/default/silos
32722024-10-17T22:41:57.360ZDEBGtest_metrics (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = f9240b89-53d4-4fad-8e28-5c7b52787427 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /v1/system/ip-pools/default/silos
32732024-10-17T22:41:57.365ZDEBGtest_metrics (dropshot_external): authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = f9240b89-53d4-4fad-8e28-5c7b52787427 resource = IpPool { parent: Fleet, key: ebef2ccd-1e4f-4648-941f-f4a7a5eb5e31, lookup_type: ByName("default") } result = Ok(()) uri = /v1/system/ip-pools/default/silos
32742024-10-17T22:41:57.373ZTRACtest_metrics (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = f9240b89-53d4-4fad-8e28-5c7b52787427 resource = Database uri = /v1/system/ip-pools/default/silos
32752024-10-17T22:41:57.373ZDEBGtest_metrics (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = f9240b89-53d4-4fad-8e28-5c7b52787427 roles = RoleSet { roles: {} } uri = /v1/system/ip-pools/default/silos
32762024-10-17T22:41:57.373ZDEBGtest_metrics (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = f9240b89-53d4-4fad-8e28-5c7b52787427 resource = Database result = Ok(()) uri = /v1/system/ip-pools/default/silos
32772024-10-17T22:41:57.373ZTRACtest_metrics (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = f9240b89-53d4-4fad-8e28-5c7b52787427 resource = Database uri = /v1/system/ip-pools/default/silos
32782024-10-17T22:41:57.373ZDEBGtest_metrics (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = f9240b89-53d4-4fad-8e28-5c7b52787427 roles = RoleSet { roles: {} } uri = /v1/system/ip-pools/default/silos
32792024-10-17T22:41:57.373ZDEBGtest_metrics (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = f9240b89-53d4-4fad-8e28-5c7b52787427 resource = Database result = Ok(()) uri = /v1/system/ip-pools/default/silos
32802024-10-17T22:41:57.373ZTRACtest_metrics (dropshot_external): authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = f9240b89-53d4-4fad-8e28-5c7b52787427 resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) } uri = /v1/system/ip-pools/default/silos
32812024-10-17T22:41:57.373ZTRACtest_metrics (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = f9240b89-53d4-4fad-8e28-5c7b52787427 resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo uri = /v1/system/ip-pools/default/silos
32822024-10-17T22:41:57.373ZTRACtest_metrics (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = f9240b89-53d4-4fad-8e28-5c7b52787427 resource = Database uri = /v1/system/ip-pools/default/silos
32832024-10-17T22:41:57.373ZDEBGtest_metrics (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = f9240b89-53d4-4fad-8e28-5c7b52787427 roles = RoleSet { roles: {} } uri = /v1/system/ip-pools/default/silos
32842024-10-17T22:41:57.373ZDEBGtest_metrics (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = f9240b89-53d4-4fad-8e28-5c7b52787427 resource = Database result = Ok(()) uri = /v1/system/ip-pools/default/silos
32852024-10-17T22:41:57.383ZTRACtest_metrics (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = f9240b89-53d4-4fad-8e28-5c7b52787427 resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet uri = /v1/system/ip-pools/default/silos
32862024-10-17T22:41:57.383ZTRACtest_metrics (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = f9240b89-53d4-4fad-8e28-5c7b52787427 resource = Database uri = /v1/system/ip-pools/default/silos
32872024-10-17T22:41:57.383ZDEBGtest_metrics (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = f9240b89-53d4-4fad-8e28-5c7b52787427 roles = RoleSet { roles: {} } uri = /v1/system/ip-pools/default/silos
32882024-10-17T22:41:57.383ZDEBGtest_metrics (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = f9240b89-53d4-4fad-8e28-5c7b52787427 resource = Database result = Ok(()) uri = /v1/system/ip-pools/default/silos
32892024-10-17T22:41:57.387ZDEBGtest_metrics (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = f9240b89-53d4-4fad-8e28-5c7b52787427 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin"), (Silo, 001de000-5110-4000-8000-000000000000, "admin")} } uri = /v1/system/ip-pools/default/silos
32902024-10-17T22:41:57.390ZDEBGtest_metrics (dropshot_external): authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = f9240b89-53d4-4fad-8e28-5c7b52787427 resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) } result = Ok(()) uri = /v1/system/ip-pools/default/silos
32912024-10-17T22:41:57.390ZTRACtest_metrics (dropshot_external): authorize begin action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = f9240b89-53d4-4fad-8e28-5c7b52787427 resource = IpPoolList uri = /v1/system/ip-pools/default/silos
32922024-10-17T22:41:57.392ZTRACtest_metrics (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = f9240b89-53d4-4fad-8e28-5c7b52787427 resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet uri = /v1/system/ip-pools/default/silos
32932024-10-17T22:41:57.392ZTRACtest_metrics (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = f9240b89-53d4-4fad-8e28-5c7b52787427 resource = Database uri = /v1/system/ip-pools/default/silos
32942024-10-17T22:41:57.392ZDEBGtest_metrics (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = f9240b89-53d4-4fad-8e28-5c7b52787427 roles = RoleSet { roles: {} } uri = /v1/system/ip-pools/default/silos
32952024-10-17T22:41:57.392ZDEBGtest_metrics (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = f9240b89-53d4-4fad-8e28-5c7b52787427 resource = Database result = Ok(()) uri = /v1/system/ip-pools/default/silos
32962024-10-17T22:41:57.397ZDEBGtest_metrics (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = f9240b89-53d4-4fad-8e28-5c7b52787427 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /v1/system/ip-pools/default/silos
32972024-10-17T22:41:57.401ZDEBGtest_metrics (dropshot_external): authorize result action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = f9240b89-53d4-4fad-8e28-5c7b52787427 resource = IpPoolList result = Ok(()) uri = /v1/system/ip-pools/default/silos
32982024-10-17T22:41:57.401ZTRACtest_metrics (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = f9240b89-53d4-4fad-8e28-5c7b52787427 resource = Database uri = /v1/system/ip-pools/default/silos
32992024-10-17T22:41:57.401ZDEBGtest_metrics (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = f9240b89-53d4-4fad-8e28-5c7b52787427 roles = RoleSet { roles: {} } uri = /v1/system/ip-pools/default/silos
33002024-10-17T22:41:57.402ZDEBGtest_metrics (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = f9240b89-53d4-4fad-8e28-5c7b52787427 resource = Database result = Ok(()) uri = /v1/system/ip-pools/default/silos
33012024-10-17T22:41:57.409ZINFOtest_metrics (dropshot_external): request completed latency_us = 97523 local_addr = 127.0.0.1:39497 method = POST remote_addr = 127.0.0.1:46292 req_id = f9240b89-53d4-4fad-8e28-5c7b52787427 response_code = 201 uri = /v1/system/ip-pools/default/silos
33022024-10-17T22:41:57.413ZINFOtest_metrics (external client test context): client received response status = 201
33032024-10-17T22:41:57.416ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34433 method = GET path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
33042024-10-17T22:41:57.417ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34433 method = POST path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
33052024-10-17T22:41:57.417ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34433 method = DELETE path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
33062024-10-17T22:41:57.417ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34433 method = GET path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
33072024-10-17T22:41:57.417ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34433 method = GET path = /crucible/0/regions/{id}/snapshots server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
33082024-10-17T22:41:57.417ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34433 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
33092024-10-17T22:41:57.417ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34433 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
33102024-10-17T22:41:57.417ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34433 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
33112024-10-17T22:41:57.417ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34433 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
33122024-10-17T22:41:57.417ZINFOtest_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:34433 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
33132024-10-17T22:41:57.417ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:34433 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
33142024-10-17T22:41:57.417ZINFOtest_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:34433 kind = storage server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
33152024-10-17T22:41:57.417ZINFOtest_metrics (nexus): upserting physical disk model = test-model physical_disk_id = 0cc86686-8850-44d3-9af7-49b732636900 serial = totally-unique-serial: 0cc86686-8850-44d3-9af7-49b732636900 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 vendor = test-vendor
33162024-10-17T22:41:57.417ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
33172024-10-17T22:41:57.417ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
33182024-10-17T22:41:57.417ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
33192024-10-17T22:41:57.418ZTRACtest_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
33202024-10-17T22:41:57.418ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
33212024-10-17T22:41:57.418ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
33222024-10-17T22:41:57.418ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
33232024-10-17T22:41:57.419ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
33242024-10-17T22:41:57.423ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33252024-10-17T22:41:57.427ZDEBGtest_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
33262024-10-17T22:41:57.600ZINFOtest_metrics (nexus): upserting zpool physical_disk_id = 0cc86686-8850-44d3-9af7-49b732636900 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 zpool_id = 384895f4-924e-4b3a-9106-10edad2a398e
33272024-10-17T22:41:57.600ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
33282024-10-17T22:41:57.600ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
33292024-10-17T22:41:57.601ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
33302024-10-17T22:41:57.602ZTRACtest_metrics (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
33312024-10-17T22:41:57.603ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
33322024-10-17T22:41:57.603ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
33332024-10-17T22:41:57.603ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
33342024-10-17T22:41:57.603ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
33352024-10-17T22:41:57.604ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33362024-10-17T22:41:57.609ZDEBGtest_metrics (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
33372024-10-17T22:41:57.626ZINFOtest_metrics (nexus): upserting dataset address = [::1]:34433 dataset_id = fbb34052-0781-43fc-85fe-240915638900 zpool_id = 384895f4-924e-4b3a-9106-10edad2a398e
33382024-10-17T22:41:57.710ZTRACtest_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
33392024-10-17T22:41:57.710ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
33402024-10-17T22:41:57.710ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
33412024-10-17T22:41:57.711ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
33422024-10-17T22:41:57.711ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
33432024-10-17T22:41:57.711ZDEBGtest_metrics (BackgroundTasks): activating background_task = inventory_collection iteration = 5 reason = Signaled
33442024-10-17T22:41:57.711ZTRACtest_metrics (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
33452024-10-17T22:41:57.711ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
33462024-10-17T22:41:57.711ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
33472024-10-17T22:41:57.711ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
33482024-10-17T22:41:57.711ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
33492024-10-17T22:41:57.713ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33502024-10-17T22:41:57.719ZDEBGtest_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
33512024-10-17T22:41:57.719ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
33522024-10-17T22:41:57.719ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
33532024-10-17T22:41:57.719ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
33542024-10-17T22:41:57.719ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33552024-10-17T22:41:57.724ZDEBGtest_metrics (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
33562024-10-17T22:41:57.724ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
33572024-10-17T22:41:57.724ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
33582024-10-17T22:41:57.725ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
33592024-10-17T22:41:57.730ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
33602024-10-17T22:41:57.731ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
33612024-10-17T22:41:57.731ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
33622024-10-17T22:41:57.754ZDEBGtest_metrics (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1b3228a8-106c-4cda-9e3a-742e41b95979, 0)]
33632024-10-17T22:41:57.755ZTRACtest_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
33642024-10-17T22:41:57.755ZTRACtest_metrics (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 16 38 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .8..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:63832 req_id = f384d1cc-6a40-4e7f-8c28-d2307ea4155f
33652024-10-17T22:41:57.755ZDEBGtest_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 5688,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63832 req_id = f384d1cc-6a40-4e7f-8c28-d2307ea4155f
33662024-10-17T22:41:57.755ZDEBGtest_metrics (store): query key key = _mgs._tcp
33672024-10-17T22:41:57.755ZDEBGtest_metrics (store): query key key = dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host
33682024-10-17T22:41:57.756ZDEBGtest_metrics (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
33692024-10-17T22:41:57.764ZDEBGtest_metrics (dns): dns response additional_records = [Record { name_labels: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:63832 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 33937, target: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal") })) }, Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 44607, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = f384d1cc-6a40-4e7f-8c28-d2307ea4155f
33702024-10-17T22:41:57.764ZDEBGtest_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 33937, target: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 44607, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2089, tv_nsec: 525522487 } })
33712024-10-17T22:41:57.764ZTRACtest_metrics (DnsResolver): lookup_service_targets: looking up SRV target
33722024-10-17T22:41:57.764ZTRACtest_metrics (DnsResolver): lookup_service_targets: looking up SRV target
33732024-10-17T22:41:57.764ZTRACtest_metrics (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: ec a4 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 30 33 39 62 65 35 36 30 2d 35 drite-039be560-5\n0020: 34 63 63 2d 34 39 65 33 2d 38 38 64 66 2d 31 61 4cc-49e3-88df-1a\n0030: 32 39 64 61 64 62 66 39 31 33 04 68 6f 73 74 0d 29dadbf913.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:59825 req_id = b0d58f03-841c-44d4-8e60-66d172c17ba8
33742024-10-17T22:41:57.764ZDEBGtest_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 60580,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 48,\n 51,\n 57,\n 98,\n 101,\n 53,\n 54,\n 48,\n 45,\n 53,\n 52,\n 99,\n 99,\n 45,\n 52,\n 57,\n 101,\n 51,\n 45,\n 56,\n 56,\n 100,\n 102,\n 45,\n 49,\n 97,\n 50,\n 57,\n 100,\n 97,\n 100,\n 98,\n 102,\n 57,\n 49,\n 51,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:59825 req_id = b0d58f03-841c-44d4-8e60-66d172c17ba8
33752024-10-17T22:41:57.764ZDEBGtest_metrics (store): query key key = dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host
33762024-10-17T22:41:57.764ZDEBGtest_metrics (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:59825 query = LowerQuery { name: LowerName(Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = b0d58f03-841c-44d4-8e60-66d172c17ba8
33772024-10-17T22:41:57.764ZTRACtest_metrics (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 44 b3 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e D...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:50672 req_id = 431393a3-eb92-409b-a408-e9d229c326c0
33782024-10-17T22:41:57.764ZDEBGtest_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 17587,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:50672 req_id = 431393a3-eb92-409b-a408-e9d229c326c0
33792024-10-17T22:41:57.764ZDEBGtest_metrics (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
33802024-10-17T22:41:57.764ZDEBGtest_metrics (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:50672 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 431393a3-eb92-409b-a408-e9d229c326c0
33812024-10-17T22:41:57.827ZTRACtest_metrics (DnsResolver): lookup_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
33822024-10-17T22:41:57.828ZTRACtest_metrics (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: cb 71 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c .q..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:54333 req_id = 3d2ffc2a-6039-4467-a9fc-eb9b2f36615c
33832024-10-17T22:41:57.828ZDEBGtest_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 52081,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54333 req_id = 3d2ffc2a-6039-4467-a9fc-eb9b2f36615c
33842024-10-17T22:41:57.828ZDEBGtest_metrics (store): query key key = _clickhouse-admin-keeper._tcp
33852024-10-17T22:41:57.828ZERROtest_metrics (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 52081, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:54333 req_id = 3d2ffc2a-6039-4467-a9fc-eb9b2f36615c
33862024-10-17T22:41:57.828ZDEBGtest_metrics (BackgroundTasks): begin collection background_task = inventory_collection
33872024-10-17T22:41:57.828ZDEBGtest_metrics (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:33937
33882024-10-17T22:41:57.828ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/ignition
33892024-10-17T22:41:57.830ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "9038a3bb-2bb9-4570-ba42-71da430bf950", "content-length": "835", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
33902024-10-17T22:41:57.830ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0
33912024-10-17T22:41:57.832ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f648d98d-7078-4480-9ed7-187ae33ae7ff", "content-length": "734", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
33922024-10-17T22:41:57.832ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0/component/sp/caboose?firmware_slot=0
33932024-10-17T22:41:57.834ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6fcda4db-caf3-4d2f-9430-cc0137de704e", "content-length": "111", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
33942024-10-17T22:41:57.834ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0/component/sp/caboose?firmware_slot=1
33952024-10-17T22:41:57.836ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "76c35e46-e6b4-4a83-863c-17b1fdc1b585", "content-length": "111", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
33962024-10-17T22:41:57.836ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0/component/rot/caboose?firmware_slot=0
33972024-10-17T22:41:57.838ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fd60a04e-b377-436d-b4cc-c8270438e845", "content-length": "167", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
33982024-10-17T22:41:57.838ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0/component/rot/caboose?firmware_slot=1
33992024-10-17T22:41:57.840ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5f90ae5d-3524-4769-8b6b-b5c5c67b85a7", "content-length": "167", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
34002024-10-17T22:41:57.841ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0/component/stage0/caboose?firmware_slot=0
34012024-10-17T22:41:57.841ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1e04621c-604d-4bbe-b460-17dddcb752b1", "content-length": "175", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
34022024-10-17T22:41:57.842ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0/component/stage0/caboose?firmware_slot=1
34032024-10-17T22:41:57.843ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a5ea197c-8f1b-4c9f-9bfe-ca5e443ccce3", "content-length": "175", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
34042024-10-17T22:41:57.844ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0/component/rot/cmpa
34052024-10-17T22:41:57.846ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ab201dc0-907f-4a54-89b4-dabbfdf60b8a", "content-length": "702", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
34062024-10-17T22:41:57.846ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0/component/rot/cfpa
34072024-10-17T22:41:57.846ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "71ae4ce3-85cc-41bd-890f-c6d9b77f37fe", "content-length": "727", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
34082024-10-17T22:41:57.846ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0/component/rot/cfpa
34092024-10-17T22:41:57.847ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "de475033-8af1-45c1-b05e-172fa497d6a7", "content-length": "729", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
34102024-10-17T22:41:57.847ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0/component/rot/cfpa
34112024-10-17T22:41:57.848ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5f4299da-2fa6-46fd-9594-99e472aacf59", "content-length": "728", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
34122024-10-17T22:41:57.848ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1
34132024-10-17T22:41:57.849ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5c482b10-0911-42a8-8629-db4e8168960d", "content-length": "734", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
34142024-10-17T22:41:57.851ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1/component/sp/caboose?firmware_slot=0
34152024-10-17T22:41:57.851ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cdc4f0ae-c8b0-4d26-bf27-431785af1842", "content-length": "111", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
34162024-10-17T22:41:57.851ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1/component/sp/caboose?firmware_slot=1
34172024-10-17T22:41:57.853ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f70323a3-2417-4f6a-909c-aae242de2885", "content-length": "111", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
34182024-10-17T22:41:57.853ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1/component/rot/caboose?firmware_slot=0
34192024-10-17T22:41:57.855ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "23a856e2-8b2b-411f-8000-29fddf1b60b9", "content-length": "167", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
34202024-10-17T22:41:57.855ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1/component/rot/caboose?firmware_slot=1
34212024-10-17T22:41:57.857ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2f31bc31-53fd-4d91-b306-d7f64889a46e", "content-length": "167", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
34222024-10-17T22:41:57.857ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1/component/stage0/caboose?firmware_slot=0
34232024-10-17T22:41:57.858ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ff69606f-4573-43b5-9f61-a0c2c44ddd47", "content-length": "175", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
34242024-10-17T22:41:57.858ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1/component/stage0/caboose?firmware_slot=1
34252024-10-17T22:41:57.860ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e4aef1c5-2529-4c8f-b244-2a90c46feb29", "content-length": "175", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
34262024-10-17T22:41:57.862ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1/component/rot/cmpa
34272024-10-17T22:41:57.862ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "25e95eaa-76a6-48c0-bed6-a1c85f3c2a2d", "content-length": "702", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
34282024-10-17T22:41:57.862ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1/component/rot/cfpa
34292024-10-17T22:41:57.862ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ecdbfff9-9f25-460c-9c8d-25439c16928f", "content-length": "727", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
34302024-10-17T22:41:57.862ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1/component/rot/cfpa
34312024-10-17T22:41:57.863ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "340ce9ac-1074-49f2-8462-962fb5468f0d", "content-length": "729", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
34322024-10-17T22:41:57.864ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1/component/rot/cfpa
34332024-10-17T22:41:57.864ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e7637fd1-b190-4fdf-b365-613d582f416e", "content-length": "728", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
34342024-10-17T22:41:57.864ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0
34352024-10-17T22:41:57.865ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "668ef61a-b327-46ff-baf6-80f3d7883102", "content-length": "723", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
34362024-10-17T22:41:57.867ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0/component/sp/caboose?firmware_slot=0
34372024-10-17T22:41:57.867ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "18f48afa-f860-4f6b-8bb0-76bbe4aee36e", "content-length": "109", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
34382024-10-17T22:41:57.867ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0/component/sp/caboose?firmware_slot=1
34392024-10-17T22:41:57.869ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b8237b62-2d6b-45bf-a0cd-239b52284629", "content-length": "109", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
34402024-10-17T22:41:57.870ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0/component/rot/caboose?firmware_slot=0
34412024-10-17T22:41:57.872ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ee64bddd-6967-4d17-8d42-92128d69d3f9", "content-length": "169", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
34422024-10-17T22:41:57.872ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0/component/rot/caboose?firmware_slot=1
34432024-10-17T22:41:57.873ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e6c8a490-918b-4692-b5e7-09ba59d8fde6", "content-length": "169", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
34442024-10-17T22:41:57.874ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0/component/stage0/caboose?firmware_slot=0
34452024-10-17T22:41:57.876ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a5f05c70-2275-460d-bd7a-f48f208eb02b", "content-length": "178", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
34462024-10-17T22:41:57.876ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0/component/stage0/caboose?firmware_slot=1
34472024-10-17T22:41:57.878ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7ea36284-e03f-4347-9326-bff6de0bcee3", "content-length": "178", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
34482024-10-17T22:41:57.878ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0/component/rot/cmpa
34492024-10-17T22:41:57.880ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f3f10215-95c8-4667-9e9d-bbf2111ea404", "content-length": "702", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
34502024-10-17T22:41:57.882ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0/component/rot/cfpa
34512024-10-17T22:41:57.882ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "eacf1854-7f62-4ea3-b869-91594c66096c", "content-length": "727", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
34522024-10-17T22:41:57.882ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0/component/rot/cfpa
34532024-10-17T22:41:57.882ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "185aaa5e-d863-446d-9ca5-cf5c86be83ce", "content-length": "729", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
34542024-10-17T22:41:57.882ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0/component/rot/cfpa
34552024-10-17T22:41:57.883ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e1306eb3-9133-42b0-84dc-6b6fc0371174", "content-length": "728", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
34562024-10-17T22:41:57.884ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1
34572024-10-17T22:41:57.884ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "393090e2-4105-447a-beae-07e9a49258e7", "content-length": "723", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
34582024-10-17T22:41:57.884ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1/component/sp/caboose?firmware_slot=0
34592024-10-17T22:41:57.886ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "54cfb6a5-415f-4f8e-a735-ecd6d0507de4", "content-length": "109", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
34602024-10-17T22:41:57.886ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1/component/sp/caboose?firmware_slot=1
34612024-10-17T22:41:57.888ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "484c460c-ba11-4d2b-af88-5b1dea5f3057", "content-length": "109", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
34622024-10-17T22:41:57.889ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1/component/rot/caboose?firmware_slot=0
34632024-10-17T22:41:57.890ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ce14d1a5-a7ce-41cb-9433-08ec0c1ba7b8", "content-length": "169", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
34642024-10-17T22:41:57.890ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1/component/rot/caboose?firmware_slot=1
34652024-10-17T22:41:57.891ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "220982f1-2828-46aa-866f-b6850955557b", "content-length": "169", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
34662024-10-17T22:41:57.891ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1/component/stage0/caboose?firmware_slot=0
34672024-10-17T22:41:57.893ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4487d1c9-7467-4b47-a4fd-dd6253d9e4ba", "content-length": "178", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
34682024-10-17T22:41:57.893ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1/component/stage0/caboose?firmware_slot=1
34692024-10-17T22:41:57.895ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2b08ac4c-d1bd-407f-af84-a3143396a1eb", "content-length": "178", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
34702024-10-17T22:41:57.895ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1/component/rot/cmpa
34712024-10-17T22:41:57.895ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1848c0b7-041e-4701-86b6-1fcd6882bfb1", "content-length": "702", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
34722024-10-17T22:41:57.896ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1/component/rot/cfpa
34732024-10-17T22:41:57.896ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b3f7e138-8c06-42f2-8362-a242985f2f7d", "content-length": "727", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
34742024-10-17T22:41:57.896ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1/component/rot/cfpa
34752024-10-17T22:41:57.897ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "83fd51da-04d8-4cf7-b3e1-ddbf6d45dd06", "content-length": "729", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
34762024-10-17T22:41:57.899ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1/component/rot/cfpa
34772024-10-17T22:41:57.899ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "60201d6f-7ada-448d-8926-23e5248044cd", "content-length": "728", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
34782024-10-17T22:41:57.899ZDEBGtest_metrics (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:44607
34792024-10-17T22:41:57.900ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:44607 method = GET uri = http://[::1]:44607/ignition
34802024-10-17T22:41:57.900ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:44607 result = Ok(Response { url: "http://[::1]:44607/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "b4638638-e3e7-46ae-8ff8-85852a984f7a", "content-length": "835", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
34812024-10-17T22:41:57.900ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:44607 method = GET uri = http://[::1]:44607/sp/switch/0
34822024-10-17T22:41:57.901ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:44607 result = Ok(Response { url: "http://[::1]:44607/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "13130ea7-14fd-47f8-bb66-e59a6d96beb7", "content-length": "734", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
34832024-10-17T22:41:57.901ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:44607 method = GET uri = http://[::1]:44607/sp/switch/1
34842024-10-17T22:41:57.902ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:44607 result = Ok(Response { url: "http://[::1]:44607/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a41c75c3-1e5f-4ab6-9e33-612a8521e423", "content-length": "734", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
34852024-10-17T22:41:57.902ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:44607 method = GET uri = http://[::1]:44607/sp/sled/0
34862024-10-17T22:41:57.903ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:44607 result = Ok(Response { url: "http://[::1]:44607/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4bc06a82-294a-43bf-9530-63e562a4910e", "content-length": "723", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
34872024-10-17T22:41:57.903ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:44607 method = GET uri = http://[::1]:44607/sp/sled/1
34882024-10-17T22:41:57.904ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:44607 result = Ok(Response { url: "http://[::1]:44607/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d0d77cea-c66d-450d-be1f-c22b9cf3010c", "content-length": "723", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
34892024-10-17T22:41:57.905ZTRACtest_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
34902024-10-17T22:41:57.906ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
34912024-10-17T22:41:57.906ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
34922024-10-17T22:41:57.906ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
34932024-10-17T22:41:57.906ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
34942024-10-17T22:41:57.907ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34952024-10-17T22:41:57.911ZDEBGtest_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
34962024-10-17T22:41:57.911ZTRACtest_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
34972024-10-17T22:41:57.912ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
34982024-10-17T22:41:57.912ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
34992024-10-17T22:41:57.912ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
35002024-10-17T22:41:57.912ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
35012024-10-17T22:41:57.913ZDEBGtest_metrics: attempt to fetch latest inventory collection result = Ok("found")
35022024-10-17T22:41:57.914ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35032024-10-17T22:41:57.919ZDEBGtest_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
35042024-10-17T22:41:57.919ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
35052024-10-17T22:41:57.919ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
35062024-10-17T22:41:57.919ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
35072024-10-17T22:41:57.952ZDEBGtest_metrics (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:35099
35082024-10-17T22:41:57.952ZDEBGtest_metrics (BackgroundTasks): client request SledAgent = http://[::1]:35099 background_task = inventory_collection body = None method = GET uri = http://[::1]:35099/inventory
35092024-10-17T22:41:57.952ZINFOtest_metrics (dropshot): accepted connection local_addr = [::1]:35099 remote_addr = [::1]:59834 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
35102024-10-17T22:41:57.953ZTRACtest_metrics (dropshot): incoming request local_addr = [::1]:35099 method = GET remote_addr = [::1]:59834 req_id = bf735db4-8f32-49dc-b92d-cd506de961d7 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /inventory
35112024-10-17T22:41:57.954ZINFOtest_metrics (dropshot): request completed latency_us = 717 local_addr = [::1]:35099 method = GET remote_addr = [::1]:59834 req_id = bf735db4-8f32-49dc-b92d-cd506de961d7 response_code = 200 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /inventory
35122024-10-17T22:41:57.955ZDEBGtest_metrics (BackgroundTasks): client response SledAgent = http://[::1]:35099 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:35099/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "bf735db4-8f32-49dc-b92d-cd506de961d7", "content-length": "4199", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
35132024-10-17T22:41:57.992ZDEBGtest_metrics (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:40953
35142024-10-17T22:41:57.993ZDEBGtest_metrics (BackgroundTasks): client request SledAgent = http://[::1]:40953 background_task = inventory_collection body = None method = GET uri = http://[::1]:40953/inventory
35152024-10-17T22:41:57.994ZTRACtest_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
35162024-10-17T22:41:57.994ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
35172024-10-17T22:41:57.994ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
35182024-10-17T22:41:57.994ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
35192024-10-17T22:41:57.994ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
35202024-10-17T22:41:57.996ZINFOtest_metrics (dropshot): accepted connection local_addr = [::1]:40953 remote_addr = [::1]:57534 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
35212024-10-17T22:41:57.996ZTRACtest_metrics (dropshot): incoming request local_addr = [::1]:40953 method = GET remote_addr = [::1]:57534 req_id = 47119775-13d6-4097-a506-53ccdfa8d5de sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
35222024-10-17T22:41:57.999ZINFOtest_metrics (dropshot): request completed latency_us = 3808 local_addr = [::1]:40953 method = GET remote_addr = [::1]:57534 req_id = 47119775-13d6-4097-a506-53ccdfa8d5de response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
35232024-10-17T22:41:58.001ZDEBGtest_metrics (BackgroundTasks): client response SledAgent = http://[::1]:40953 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:40953/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "47119775-13d6-4097-a506-53ccdfa8d5de", "content-length": "6815", "date": "Thu, 17 Oct 2024 22:41:57 GMT"} })
35242024-10-17T22:41:58.003ZDEBGtest_metrics (BackgroundTasks): finished collection background_task = inventory_collection
35252024-10-17T22:41:58.006ZTRACtest_metrics (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
35262024-10-17T22:41:58.006ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
35272024-10-17T22:41:58.006ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
35282024-10-17T22:41:58.006ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
35292024-10-17T22:41:58.007ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
35302024-10-17T22:41:58.008ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35312024-10-17T22:41:58.014ZDEBGtest_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
35322024-10-17T22:41:58.014ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
35332024-10-17T22:41:58.014ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
35342024-10-17T22:41:58.014ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
35352024-10-17T22:41:58.020ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35362024-10-17T22:41:58.024ZDEBGtest_metrics (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
35372024-10-17T22:41:58.025ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
35382024-10-17T22:41:58.025ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
35392024-10-17T22:41:58.025ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
35402024-10-17T22:41:58.033ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
35412024-10-17T22:41:58.035ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
35422024-10-17T22:41:58.035ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
35432024-10-17T22:41:58.085ZDEBGtest_metrics: attempt to fetch latest inventory collection result = Ok("found")
35442024-10-17T22:41:58.137ZTRACtest_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
35452024-10-17T22:41:58.138ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
35462024-10-17T22:41:58.138ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
35472024-10-17T22:41:58.138ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
35482024-10-17T22:41:58.138ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
35492024-10-17T22:41:58.139ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35502024-10-17T22:41:58.144ZDEBGtest_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
35512024-10-17T22:41:58.145ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
35522024-10-17T22:41:58.145ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
35532024-10-17T22:41:58.145ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
35542024-10-17T22:41:58.186ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
35552024-10-17T22:41:58.190ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
35562024-10-17T22:41:58.243ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
35572024-10-17T22:41:58.243ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
35582024-10-17T22:41:58.244ZDEBGtest_metrics (clickhouse-client): retrieving timeseries schema from database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
35592024-10-17T22:41:58.250ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = SELECT * FROM oximeter.timeseries_schema FORMAT JSONEachRow;
35602024-10-17T22:41:58.255ZTRACtest_metrics (clickhouse-client): no new timeseries schema in database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
35612024-10-17T22:41:58.255ZDEBGtest_metrics (clickhouse-client): new timeseries schema collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 schema = {"timeseries_name":"integration_target:integration_metric","fields.name":["metric_name","target_name"],"fields.type":["String","String"],"fields.source":["Metric","Target"],"datum_type":"I64","created":"2024-10-17 22:41:58.244559519"} timeseries_name = integration_target:integration_metric
35622024-10-17T22:41:58.255ZDEBGtest_metrics (clickhouse-client): inserting 1 new timeseries schema collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
35632024-10-17T22:41:58.257ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.timeseries_schema FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","fields.name":["metric_name","target_name"],"fields.type":["String","String"],"fields.source":["Metric","Target"],"datum_type":"I64","created":"2024-10-17 22:41:58.244559519"}\n
35642024-10-17T22:41:58.257ZINFOtest_metrics (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 4cdc8bc0-acad-463c-aa93-04376cc0e9a4
35652024-10-17T22:41:58.257ZDEBGtest_metrics (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 4cdc8bc0-acad-463c-aa93-04376cc0e9a4 time_started = 2024-10-17 22:41:57.828671 UTC
35662024-10-17T22:41:58.258ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 546.880291ms iteration = 5 status = {"collection_id":"4cdc8bc0-acad-463c-aa93-04376cc0e9a4","time_done":"2024-10-17 22:41:58.003812 UTC","time_started":"2024-10-17 22:41:57.828671 UTC"}
35672024-10-17T22:41:58.258ZDEBGtest_metrics (BackgroundTasks): activating background_task = inventory_collection iteration = 6 reason = Signaled
35682024-10-17T22:41:58.258ZTRACtest_metrics (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
35692024-10-17T22:41:58.258ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
35702024-10-17T22:41:58.258ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
35712024-10-17T22:41:58.258ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
35722024-10-17T22:41:58.258ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
35732024-10-17T22:41:58.258ZDEBGtest_metrics (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 3 reason = Dependency
35742024-10-17T22:41:58.258ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 40.081\u{b5}s iteration = 3 status = {"error":"task disabled"}
35752024-10-17T22:41:58.259ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35762024-10-17T22:41:58.263ZDEBGtest_metrics (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
35772024-10-17T22:41:58.263ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
35782024-10-17T22:41:58.263ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
35792024-10-17T22:41:58.263ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
35802024-10-17T22:41:58.264ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
35812024-10-17T22:41:58.274ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
35822024-10-17T22:41:58.274ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:41:58.187741376","datum":0}\n
35832024-10-17T22:41:58.276ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
35842024-10-17T22:41:58.276ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
35852024-10-17T22:41:58.310ZDEBGtest_metrics (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1b3228a8-106c-4cda-9e3a-742e41b95979, 0), (4cdc8bc0-acad-463c-aa93-04376cc0e9a4, 0)]
35862024-10-17T22:41:58.310ZTRACtest_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
35872024-10-17T22:41:58.310ZTRACtest_metrics (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: ac a6 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:58067 req_id = 5439152e-bbb8-4ab6-a341-dcc72459cf23
35882024-10-17T22:41:58.310ZDEBGtest_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 44198,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:58067 req_id = 5439152e-bbb8-4ab6-a341-dcc72459cf23
35892024-10-17T22:41:58.311ZDEBGtest_metrics (store): query key key = _mgs._tcp
35902024-10-17T22:41:58.311ZDEBGtest_metrics (store): query key key = dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host
35912024-10-17T22:41:58.311ZDEBGtest_metrics (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
35922024-10-17T22:41:58.311ZDEBGtest_metrics (dns): dns response additional_records = [Record { name_labels: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:58067 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 33937, target: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal") })) }, Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 44607, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 5439152e-bbb8-4ab6-a341-dcc72459cf23
35932024-10-17T22:41:58.312ZDEBGtest_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 33937, target: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 44607, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2090, tv_nsec: 80961573 } })
35942024-10-17T22:41:58.312ZTRACtest_metrics (DnsResolver): lookup_service_targets: looking up SRV target
35952024-10-17T22:41:58.312ZTRACtest_metrics (DnsResolver): lookup_service_targets: looking up SRV target
35962024-10-17T22:41:58.312ZTRACtest_metrics (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: ad 21 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .!..........-den\n0010: 64 72 69 74 65 2d 30 33 39 62 65 35 36 30 2d 35 drite-039be560-5\n0020: 34 63 63 2d 34 39 65 33 2d 38 38 64 66 2d 31 61 4cc-49e3-88df-1a\n0030: 32 39 64 61 64 62 66 39 31 33 04 68 6f 73 74 0d 29dadbf913.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:61856 req_id = 137b5c31-90cb-4f3d-8f65-eef54cc72e93
35972024-10-17T22:41:58.312ZDEBGtest_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 44321,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 48,\n 51,\n 57,\n 98,\n 101,\n 53,\n 54,\n 48,\n 45,\n 53,\n 52,\n 99,\n 99,\n 45,\n 52,\n 57,\n 101,\n 51,\n 45,\n 56,\n 56,\n 100,\n 102,\n 45,\n 49,\n 97,\n 50,\n 57,\n 100,\n 97,\n 100,\n 98,\n 102,\n 57,\n 49,\n 51,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:61856 req_id = 137b5c31-90cb-4f3d-8f65-eef54cc72e93
35982024-10-17T22:41:58.313ZDEBGtest_metrics (store): query key key = dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host
35992024-10-17T22:41:58.313ZDEBGtest_metrics (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:61856 query = LowerQuery { name: LowerName(Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 137b5c31-90cb-4f3d-8f65-eef54cc72e93
36002024-10-17T22:41:58.313ZTRACtest_metrics (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 8a 9d 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:61370 req_id = 0493d7c5-8029-4581-909f-5365d4a3acd8
36012024-10-17T22:41:58.313ZDEBGtest_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 35485,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:61370 req_id = 0493d7c5-8029-4581-909f-5365d4a3acd8
36022024-10-17T22:41:58.313ZDEBGtest_metrics (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
36032024-10-17T22:41:58.314ZDEBGtest_metrics (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:61370 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 0493d7c5-8029-4581-909f-5365d4a3acd8
36042024-10-17T22:41:58.368ZTRACtest_metrics (DnsResolver): lookup_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
36052024-10-17T22:41:58.368ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
36062024-10-17T22:41:58.369ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
36072024-10-17T22:41:58.369ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
36082024-10-17T22:41:58.369ZTRACtest_metrics (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 1c f1 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:49406 req_id = 93dbf715-6266-49ba-93d8-41b0df05983c
36092024-10-17T22:41:58.370ZDEBGtest_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 7409,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:49406 req_id = 93dbf715-6266-49ba-93d8-41b0df05983c
36102024-10-17T22:41:58.370ZDEBGtest_metrics (store): query key key = _clickhouse-admin-keeper._tcp
36112024-10-17T22:41:58.370ZERROtest_metrics (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 7409, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:49406 req_id = 93dbf715-6266-49ba-93d8-41b0df05983c
36122024-10-17T22:41:58.370ZDEBGtest_metrics (BackgroundTasks): begin collection background_task = inventory_collection
36132024-10-17T22:41:58.370ZDEBGtest_metrics (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:33937
36142024-10-17T22:41:58.370ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/ignition
36152024-10-17T22:41:58.372ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "88833717-ea9c-4864-8b98-3670d20a4dd8", "content-length": "835", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
36162024-10-17T22:41:58.376ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0
36172024-10-17T22:41:58.376ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f082238d-ee7b-43ce-a744-64d01cf9a442", "content-length": "734", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
36182024-10-17T22:41:58.376ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0/component/sp/caboose?firmware_slot=0
36192024-10-17T22:41:58.376ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9a4ee892-d948-4353-a84a-99c1ed1d505d", "content-length": "111", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
36202024-10-17T22:41:58.376ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0/component/sp/caboose?firmware_slot=1
36212024-10-17T22:41:58.378ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "26cc259b-fcbc-469f-a911-922fd55a7db8", "content-length": "111", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
36222024-10-17T22:41:58.378ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0/component/rot/caboose?firmware_slot=0
36232024-10-17T22:41:58.379ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a20b3965-fcd1-4473-b5bc-31d92d8c4bae", "content-length": "167", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
36242024-10-17T22:41:58.380ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0/component/rot/caboose?firmware_slot=1
36252024-10-17T22:41:58.381ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "955834d1-e857-48a8-bb5f-e38e626c00f4", "content-length": "167", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
36262024-10-17T22:41:58.382ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0/component/stage0/caboose?firmware_slot=0
36272024-10-17T22:41:58.383ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ed8e5031-f7b8-4a35-8f46-feb6f6da09dc", "content-length": "175", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
36282024-10-17T22:41:58.384ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0/component/stage0/caboose?firmware_slot=1
36292024-10-17T22:41:58.385ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2ee98e13-19c4-45ab-aa83-8471f891d849", "content-length": "175", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
36302024-10-17T22:41:58.387ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0/component/rot/cmpa
36312024-10-17T22:41:58.387ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5ad61b8f-6377-436b-a73c-5efa5c9dd2b3", "content-length": "702", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
36322024-10-17T22:41:58.387ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0/component/rot/cfpa
36332024-10-17T22:41:58.387ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b01aa1be-58fd-472f-97c0-2e0a0f70b3b0", "content-length": "727", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
36342024-10-17T22:41:58.387ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0/component/rot/cfpa
36352024-10-17T22:41:58.389ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b452c49a-e556-4472-978c-402c72d1af8d", "content-length": "729", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
36362024-10-17T22:41:58.389ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0/component/rot/cfpa
36372024-10-17T22:41:58.390ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f2e1f3c8-12b5-46a4-90d8-5b218a5aa7b0", "content-length": "728", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
36382024-10-17T22:41:58.390ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1
36392024-10-17T22:41:58.391ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "06d5f4a8-d0b5-4fa7-a8a1-f742d167c855", "content-length": "734", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
36402024-10-17T22:41:58.392ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1/component/sp/caboose?firmware_slot=0
36412024-10-17T22:41:58.393ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3738d1e4-10f2-4e7d-8328-1cf3fa1845d0", "content-length": "111", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
36422024-10-17T22:41:58.393ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1/component/sp/caboose?firmware_slot=1
36432024-10-17T22:41:58.395ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6a22bec1-b7d8-496c-b10c-0b28d171daaa", "content-length": "111", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
36442024-10-17T22:41:58.396ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1/component/rot/caboose?firmware_slot=0
36452024-10-17T22:41:58.397ZDEBGtest_metrics: attempt to fetch latest inventory collection result = Ok("found")
36462024-10-17T22:41:58.398ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38703 method = GET path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
36472024-10-17T22:41:58.399ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38703 method = POST path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
36482024-10-17T22:41:58.399ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38703 method = DELETE path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
36492024-10-17T22:41:58.399ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38703 method = GET path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
36502024-10-17T22:41:58.399ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38703 method = GET path = /crucible/0/regions/{id}/snapshots server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
36512024-10-17T22:41:58.399ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38703 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
36522024-10-17T22:41:58.399ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38703 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
36532024-10-17T22:41:58.399ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38703 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
36542024-10-17T22:41:58.399ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38703 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
36552024-10-17T22:41:58.399ZINFOtest_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:38703 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
36562024-10-17T22:41:58.399ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:38703 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
36572024-10-17T22:41:58.399ZINFOtest_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:38703 kind = storage server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
36582024-10-17T22:41:58.399ZINFOtest_metrics (nexus): upserting physical disk model = test-model physical_disk_id = 055b9cc3-5d00-4cf3-b9b1-97b618c7324e serial = totally-unique-serial: 055b9cc3-5d00-4cf3-b9b1-97b618c7324e sled_id = b6d65341-167c-41df-9b5c-41cded99c229 vendor = test-vendor
36592024-10-17T22:41:58.399ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
36602024-10-17T22:41:58.399ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
36612024-10-17T22:41:58.400ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
36622024-10-17T22:41:58.400ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ca4213fc-d632-48cd-98ba-ab01c3b0ce54", "content-length": "167", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
36632024-10-17T22:41:58.401ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1/component/rot/caboose?firmware_slot=1
36642024-10-17T22:41:58.401ZTRACtest_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
36652024-10-17T22:41:58.401ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
36662024-10-17T22:41:58.401ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
36672024-10-17T22:41:58.401ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
36682024-10-17T22:41:58.401ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
36692024-10-17T22:41:58.402ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36702024-10-17T22:41:58.406ZDEBGtest_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
36712024-10-17T22:41:58.406ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "80707964-6030-481d-b1d6-d90911d54567", "content-length": "167", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
36722024-10-17T22:41:58.407ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1/component/stage0/caboose?firmware_slot=0
36732024-10-17T22:41:58.409ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "dd73af32-bab2-41d4-a850-209bb2c53478", "content-length": "175", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
36742024-10-17T22:41:58.409ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1/component/stage0/caboose?firmware_slot=1
36752024-10-17T22:41:58.410ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4a8ed94e-56f7-484f-ab1f-e9471ac1fa00", "content-length": "175", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
36762024-10-17T22:41:58.411ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1/component/rot/cmpa
36772024-10-17T22:41:58.419ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "442b2d09-2e1f-488c-93e6-67af87b7f8dc", "content-length": "702", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
36782024-10-17T22:41:58.419ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1/component/rot/cfpa
36792024-10-17T22:41:58.419ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "906b7eeb-439b-4fe4-9d85-4cc05c550e3e", "content-length": "727", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
36802024-10-17T22:41:58.419ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1/component/rot/cfpa
36812024-10-17T22:41:58.419ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "656f21fc-3f8b-4e3b-9bd8-8f62500b8ce0", "content-length": "729", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
36822024-10-17T22:41:58.419ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1/component/rot/cfpa
36832024-10-17T22:41:58.419ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9d5e3213-32c3-479a-a3e1-1ea206873a6e", "content-length": "728", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
36842024-10-17T22:41:58.420ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0
36852024-10-17T22:41:58.420ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e6e326a8-020b-41eb-b0c8-96b6e90bef3f", "content-length": "723", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
36862024-10-17T22:41:58.420ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0/component/sp/caboose?firmware_slot=0
36872024-10-17T22:41:58.420ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3dabadd8-e37f-4753-a9a3-b1b4e9c9c456", "content-length": "109", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
36882024-10-17T22:41:58.421ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0/component/sp/caboose?firmware_slot=1
36892024-10-17T22:41:58.421ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "74070ce0-fbc8-44cd-95bf-887ab8fac557", "content-length": "109", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
36902024-10-17T22:41:58.421ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0/component/rot/caboose?firmware_slot=0
36912024-10-17T22:41:58.421ZINFOtest_metrics (nexus): upserting zpool physical_disk_id = 055b9cc3-5d00-4cf3-b9b1-97b618c7324e sled_id = b6d65341-167c-41df-9b5c-41cded99c229 zpool_id = d0a88f7f-d9cf-4935-90f2-4e03940a5b1b
36922024-10-17T22:41:58.421ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
36932024-10-17T22:41:58.421ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
36942024-10-17T22:41:58.421ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
36952024-10-17T22:41:58.421ZTRACtest_metrics (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
36962024-10-17T22:41:58.422ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
36972024-10-17T22:41:58.422ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
36982024-10-17T22:41:58.422ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
36992024-10-17T22:41:58.423ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
37002024-10-17T22:41:58.423ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "831e1fc3-5fc0-4325-a689-a17d1e41db73", "content-length": "169", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
37012024-10-17T22:41:58.424ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0/component/rot/caboose?firmware_slot=1
37022024-10-17T22:41:58.424ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37032024-10-17T22:41:58.426ZDEBGtest_metrics (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
37042024-10-17T22:41:58.428ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4e8bf26b-efde-496e-9c9a-ae08aa96ef6b", "content-length": "169", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
37052024-10-17T22:41:58.428ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0/component/stage0/caboose?firmware_slot=0
37062024-10-17T22:41:58.431ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "97aa5c69-a4f3-48f3-9722-c583e3e55e03", "content-length": "178", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
37072024-10-17T22:41:58.431ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0/component/stage0/caboose?firmware_slot=1
37082024-10-17T22:41:58.432ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "faae8cb3-2f80-463a-b594-4eeafb800163", "content-length": "178", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
37092024-10-17T22:41:58.432ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0/component/rot/cmpa
37102024-10-17T22:41:58.433ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3cc8430c-4867-467f-af87-9d225072e791", "content-length": "702", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
37112024-10-17T22:41:58.433ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0/component/rot/cfpa
37122024-10-17T22:41:58.434ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0285abb2-2d49-49b8-b859-6eb5257276f0", "content-length": "727", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
37132024-10-17T22:41:58.435ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0/component/rot/cfpa
37142024-10-17T22:41:58.435ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2ea8e138-cb36-479a-8751-e68b0993815a", "content-length": "729", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
37152024-10-17T22:41:58.435ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0/component/rot/cfpa
37162024-10-17T22:41:58.436ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b79d6e06-db7e-491e-84e1-b32d294cdb65", "content-length": "728", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
37172024-10-17T22:41:58.436ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1
37182024-10-17T22:41:58.437ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a96e14e7-0e8c-441c-823f-a0ea82e9838a", "content-length": "723", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
37192024-10-17T22:41:58.437ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1/component/sp/caboose?firmware_slot=0
37202024-10-17T22:41:58.439ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6bebcf11-6ab8-4352-8233-21ec34fd3a24", "content-length": "109", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
37212024-10-17T22:41:58.439ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1/component/sp/caboose?firmware_slot=1
37222024-10-17T22:41:58.441ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d48a7630-2fb5-4720-9bb2-d4881892feee", "content-length": "109", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
37232024-10-17T22:41:58.442ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1/component/rot/caboose?firmware_slot=0
37242024-10-17T22:41:58.443ZINFOtest_metrics (nexus): upserting dataset address = [::1]:38703 dataset_id = 8f180f7b-4cfa-4e42-9cbf-2fb1ec151127 zpool_id = d0a88f7f-d9cf-4935-90f2-4e03940a5b1b
37252024-10-17T22:41:58.443ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "80fcdef7-d6f2-4947-9c40-6af0e3147c57", "content-length": "169", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
37262024-10-17T22:41:58.443ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1/component/rot/caboose?firmware_slot=1
37272024-10-17T22:41:58.445ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9d6a5570-b6dd-4ed2-9fff-e00921623f5e", "content-length": "169", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
37282024-10-17T22:41:58.445ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1/component/stage0/caboose?firmware_slot=0
37292024-10-17T22:41:58.447ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f4a0a8e-1e16-4cc5-8e65-709a130934f1", "content-length": "178", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
37302024-10-17T22:41:58.453ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1/component/stage0/caboose?firmware_slot=1
37312024-10-17T22:41:58.453ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d9dd688b-dac0-4788-98dc-cab320ab1ad1", "content-length": "178", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
37322024-10-17T22:41:58.453ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1/component/rot/cmpa
37332024-10-17T22:41:58.453ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f5367d7c-13cc-4b83-85bc-bc266b28f4b2", "content-length": "702", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
37342024-10-17T22:41:58.453ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1/component/rot/cfpa
37352024-10-17T22:41:58.453ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a9cfbe0-35e3-40e1-9924-37c58c7e6bce", "content-length": "727", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
37362024-10-17T22:41:58.453ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1/component/rot/cfpa
37372024-10-17T22:41:58.454ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4cd39fef-fc8c-4d19-b6c1-e631298fc501", "content-length": "729", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
37382024-10-17T22:41:58.454ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1/component/rot/cfpa
37392024-10-17T22:41:58.454ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5921df3c-7219-442e-a7d1-aa121919ed8d", "content-length": "728", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
37402024-10-17T22:41:58.454ZDEBGtest_metrics (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:44607
37412024-10-17T22:41:58.454ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:44607 method = GET uri = http://[::1]:44607/ignition
37422024-10-17T22:41:58.457ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:44607 result = Ok(Response { url: "http://[::1]:44607/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "969afd8c-e5e9-4c87-b487-c8857bfb9301", "content-length": "835", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
37432024-10-17T22:41:58.460ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:44607 method = GET uri = http://[::1]:44607/sp/switch/0
37442024-10-17T22:41:58.460ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:44607 result = Ok(Response { url: "http://[::1]:44607/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b6ccc62b-de88-45c8-a1c2-e0c3a1570d82", "content-length": "734", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
37452024-10-17T22:41:58.460ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:44607 method = GET uri = http://[::1]:44607/sp/switch/1
37462024-10-17T22:41:58.460ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:44607 result = Ok(Response { url: "http://[::1]:44607/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "16ff4de7-75e7-4ef9-a355-c1d815470a86", "content-length": "734", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
37472024-10-17T22:41:58.460ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:44607 method = GET uri = http://[::1]:44607/sp/sled/0
37482024-10-17T22:41:58.460ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:44607 result = Ok(Response { url: "http://[::1]:44607/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5e2144d8-75b4-4fdd-9b0e-106d25d7940a", "content-length": "723", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
37492024-10-17T22:41:58.460ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:44607 method = GET uri = http://[::1]:44607/sp/sled/1
37502024-10-17T22:41:58.460ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:44607 result = Ok(Response { url: "http://[::1]:44607/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3af1070a-bf48-4613-a603-fbc2b82852eb", "content-length": "723", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
37512024-10-17T22:41:58.460ZTRACtest_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
37522024-10-17T22:41:58.460ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
37532024-10-17T22:41:58.461ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
37542024-10-17T22:41:58.461ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
37552024-10-17T22:41:58.461ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
37562024-10-17T22:41:58.468ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37572024-10-17T22:41:58.468ZDEBGtest_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
37582024-10-17T22:41:58.468ZTRACtest_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
37592024-10-17T22:41:58.468ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
37602024-10-17T22:41:58.468ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
37612024-10-17T22:41:58.469ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
37622024-10-17T22:41:58.469ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
37632024-10-17T22:41:58.470ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37642024-10-17T22:41:58.476ZDEBGtest_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
37652024-10-17T22:41:58.476ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
37662024-10-17T22:41:58.476ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
37672024-10-17T22:41:58.477ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
37682024-10-17T22:41:58.477ZTRACtest_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
37692024-10-17T22:41:58.478ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
37702024-10-17T22:41:58.478ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
37712024-10-17T22:41:58.478ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
37722024-10-17T22:41:58.478ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
37732024-10-17T22:41:58.512ZDEBGtest_metrics (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:35099
37742024-10-17T22:41:58.514ZDEBGtest_metrics (BackgroundTasks): client request SledAgent = http://[::1]:35099 background_task = inventory_collection body = None method = GET uri = http://[::1]:35099/inventory
37752024-10-17T22:41:58.514ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37762024-10-17T22:41:58.518ZDEBGtest_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
37772024-10-17T22:41:58.518ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
37782024-10-17T22:41:58.518ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
37792024-10-17T22:41:58.519ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
37802024-10-17T22:41:58.519ZINFOtest_metrics (dropshot): accepted connection local_addr = [::1]:35099 remote_addr = [::1]:59836 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
37812024-10-17T22:41:58.519ZTRACtest_metrics (dropshot): incoming request local_addr = [::1]:35099 method = GET remote_addr = [::1]:59836 req_id = 1fd793cc-d2d6-4a70-9309-d0374dcb4c84 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /inventory
37822024-10-17T22:41:58.520ZINFOtest_metrics (dropshot): request completed latency_us = 455 local_addr = [::1]:35099 method = GET remote_addr = [::1]:59836 req_id = 1fd793cc-d2d6-4a70-9309-d0374dcb4c84 response_code = 200 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /inventory
37832024-10-17T22:41:58.520ZDEBGtest_metrics (BackgroundTasks): client response SledAgent = http://[::1]:35099 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:35099/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "1fd793cc-d2d6-4a70-9309-d0374dcb4c84", "content-length": "4199", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
37842024-10-17T22:41:58.556ZDEBGtest_metrics (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:40953
37852024-10-17T22:41:58.557ZDEBGtest_metrics (BackgroundTasks): client request SledAgent = http://[::1]:40953 background_task = inventory_collection body = None method = GET uri = http://[::1]:40953/inventory
37862024-10-17T22:41:58.557ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
37872024-10-17T22:41:58.557ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
37882024-10-17T22:41:58.558ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
37892024-10-17T22:41:58.558ZINFOtest_metrics (dropshot): accepted connection local_addr = [::1]:40953 remote_addr = [::1]:57536 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
37902024-10-17T22:41:58.559ZTRACtest_metrics (dropshot): incoming request local_addr = [::1]:40953 method = GET remote_addr = [::1]:57536 req_id = 7ac80d4d-75ce-43e2-b9fc-f9c854605ab8 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
37912024-10-17T22:41:58.559ZINFOtest_metrics (dropshot): request completed latency_us = 598 local_addr = [::1]:40953 method = GET remote_addr = [::1]:57536 req_id = 7ac80d4d-75ce-43e2-b9fc-f9c854605ab8 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
37922024-10-17T22:41:58.560ZDEBGtest_metrics (BackgroundTasks): client response SledAgent = http://[::1]:40953 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:40953/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "7ac80d4d-75ce-43e2-b9fc-f9c854605ab8", "content-length": "7190", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
37932024-10-17T22:41:58.561ZDEBGtest_metrics (BackgroundTasks): finished collection background_task = inventory_collection
37942024-10-17T22:41:58.563ZTRACtest_metrics (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
37952024-10-17T22:41:58.563ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
37962024-10-17T22:41:58.564ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
37972024-10-17T22:41:58.564ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
37982024-10-17T22:41:58.565ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
37992024-10-17T22:41:58.581ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38002024-10-17T22:41:58.586ZDEBGtest_metrics (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
38012024-10-17T22:41:58.589ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
38022024-10-17T22:41:58.589ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
38032024-10-17T22:41:58.589ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
38042024-10-17T22:41:58.619ZDEBGtest_metrics: attempt to fetch latest inventory collection result = Ok("found")
38052024-10-17T22:41:58.672ZTRACtest_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
38062024-10-17T22:41:58.672ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
38072024-10-17T22:41:58.672ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
38082024-10-17T22:41:58.672ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
38092024-10-17T22:41:58.673ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
38102024-10-17T22:41:58.676ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38112024-10-17T22:41:58.682ZDEBGtest_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
38122024-10-17T22:41:58.683ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
38132024-10-17T22:41:58.683ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
38142024-10-17T22:41:58.683ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
38152024-10-17T22:41:58.908ZINFOtest_metrics (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 5feb46a1-e3a3-4c4e-bfbb-6ff6d63eb732
38162024-10-17T22:41:58.909ZDEBGtest_metrics (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 5feb46a1-e3a3-4c4e-bfbb-6ff6d63eb732 time_started = 2024-10-17 22:41:58.370625 UTC
38172024-10-17T22:41:58.909ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 651.586272ms iteration = 6 status = {"collection_id":"5feb46a1-e3a3-4c4e-bfbb-6ff6d63eb732","time_done":"2024-10-17 22:41:58.561853 UTC","time_started":"2024-10-17 22:41:58.370625 UTC"}
38182024-10-17T22:41:58.909ZDEBGtest_metrics (BackgroundTasks): activating background_task = inventory_collection iteration = 7 reason = Signaled
38192024-10-17T22:41:58.909ZTRACtest_metrics (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
38202024-10-17T22:41:58.909ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
38212024-10-17T22:41:58.909ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
38222024-10-17T22:41:58.909ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
38232024-10-17T22:41:58.910ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
38242024-10-17T22:41:58.910ZDEBGtest_metrics (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 4 reason = Dependency
38252024-10-17T22:41:58.910ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 64.532\u{b5}s iteration = 4 status = {"error":"task disabled"}
38262024-10-17T22:41:58.911ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38272024-10-17T22:41:58.915ZDEBGtest_metrics (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
38282024-10-17T22:41:58.915ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
38292024-10-17T22:41:58.915ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
38302024-10-17T22:41:58.915ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
38312024-10-17T22:41:58.919ZDEBGtest_metrics (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1b3228a8-106c-4cda-9e3a-742e41b95979, 0), (4cdc8bc0-acad-463c-aa93-04376cc0e9a4, 0), (5feb46a1-e3a3-4c4e-bfbb-6ff6d63eb732, 0)]
38322024-10-17T22:41:58.920ZTRACtest_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
38332024-10-17T22:41:58.920ZTRACtest_metrics (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 03 75 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .u..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:52548 req_id = 0033394c-72b8-44ba-b694-c44266505251
38342024-10-17T22:41:58.920ZDEBGtest_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 885,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52548 req_id = 0033394c-72b8-44ba-b694-c44266505251
38352024-10-17T22:41:58.920ZDEBGtest_metrics (store): query key key = _mgs._tcp
38362024-10-17T22:41:58.921ZDEBGtest_metrics (store): query key key = dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host
38372024-10-17T22:41:58.921ZDEBGtest_metrics (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
38382024-10-17T22:41:58.921ZDEBGtest_metrics (dns): dns response additional_records = [Record { name_labels: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:52548 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 33937, target: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal") })) }, Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 44607, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 0033394c-72b8-44ba-b694-c44266505251
38392024-10-17T22:41:58.921ZDEBGtest_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 33937, target: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 44607, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2090, tv_nsec: 690572267 } })
38402024-10-17T22:41:58.921ZTRACtest_metrics (DnsResolver): lookup_service_targets: looking up SRV target
38412024-10-17T22:41:58.921ZTRACtest_metrics (DnsResolver): lookup_service_targets: looking up SRV target
38422024-10-17T22:41:58.922ZTRACtest_metrics (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 27 69 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e 'i..........-den\n0010: 64 72 69 74 65 2d 30 33 39 62 65 35 36 30 2d 35 drite-039be560-5\n0020: 34 63 63 2d 34 39 65 33 2d 38 38 64 66 2d 31 61 4cc-49e3-88df-1a\n0030: 32 39 64 61 64 62 66 39 31 33 04 68 6f 73 74 0d 29dadbf913.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:51526 req_id = 45fd7063-dfde-44eb-9348-477e2f4d8628
38432024-10-17T22:41:58.922ZDEBGtest_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 10089,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 48,\n 51,\n 57,\n 98,\n 101,\n 53,\n 54,\n 48,\n 45,\n 53,\n 52,\n 99,\n 99,\n 45,\n 52,\n 57,\n 101,\n 51,\n 45,\n 56,\n 56,\n 100,\n 102,\n 45,\n 49,\n 97,\n 50,\n 57,\n 100,\n 97,\n 100,\n 98,\n 102,\n 57,\n 49,\n 51,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51526 req_id = 45fd7063-dfde-44eb-9348-477e2f4d8628
38442024-10-17T22:41:58.923ZDEBGtest_metrics (store): query key key = dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host
38452024-10-17T22:41:58.923ZDEBGtest_metrics (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:51526 query = LowerQuery { name: LowerName(Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 45fd7063-dfde-44eb-9348-477e2f4d8628
38462024-10-17T22:41:58.923ZTRACtest_metrics (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 90 31 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .1..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:51929 req_id = e5aca2db-a990-43b4-8fdf-f0a24ab9fdcf
38472024-10-17T22:41:58.923ZDEBGtest_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 36913,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51929 req_id = e5aca2db-a990-43b4-8fdf-f0a24ab9fdcf
38482024-10-17T22:41:58.923ZDEBGtest_metrics (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
38492024-10-17T22:41:58.923ZDEBGtest_metrics (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:51929 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = e5aca2db-a990-43b4-8fdf-f0a24ab9fdcf
38502024-10-17T22:41:58.998ZTRACtest_metrics (DnsResolver): lookup_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
38512024-10-17T22:41:58.999ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
38522024-10-17T22:41:58.999ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
38532024-10-17T22:41:58.999ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
38542024-10-17T22:41:59.003ZTRACtest_metrics (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: f4 28 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c .(..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:64777 req_id = 124032ea-260a-4ae7-9b39-eb26b3e80416
38552024-10-17T22:41:59.003ZDEBGtest_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 62504,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64777 req_id = 124032ea-260a-4ae7-9b39-eb26b3e80416
38562024-10-17T22:41:59.003ZDEBGtest_metrics (store): query key key = _clickhouse-admin-keeper._tcp
38572024-10-17T22:41:59.003ZERROtest_metrics (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 62504, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:64777 req_id = 124032ea-260a-4ae7-9b39-eb26b3e80416
38582024-10-17T22:41:59.004ZDEBGtest_metrics (BackgroundTasks): begin collection background_task = inventory_collection
38592024-10-17T22:41:59.006ZDEBGtest_metrics (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:33937
38602024-10-17T22:41:59.006ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/ignition
38612024-10-17T22:41:59.011ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "9e3fb658-d688-4320-8385-8f46e67e2f29", "content-length": "835", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
38622024-10-17T22:41:59.012ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0
38632024-10-17T22:41:59.026ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "dcfceaba-1c16-47b6-b929-9e90290f6d83", "content-length": "734", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
38642024-10-17T22:41:59.027ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0/component/sp/caboose?firmware_slot=0
38652024-10-17T22:41:59.039ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9595db60-4ab4-4a4f-ae11-8a3890062c20", "content-length": "111", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
38662024-10-17T22:41:59.039ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0/component/sp/caboose?firmware_slot=1
38672024-10-17T22:41:59.044ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4b8f5850-9f6d-4171-838c-8ebb9c73b663", "content-length": "111", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
38682024-10-17T22:41:59.044ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0/component/rot/caboose?firmware_slot=0
38692024-10-17T22:41:59.047ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "19e52f1b-b4ff-4e0a-9220-c0ea870446ca", "content-length": "167", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
38702024-10-17T22:41:59.047ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0/component/rot/caboose?firmware_slot=1
38712024-10-17T22:41:59.048ZDEBGtest_metrics: attempt to fetch latest inventory collection result = Ok("found")
38722024-10-17T22:41:59.049ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43317 method = GET path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
38732024-10-17T22:41:59.050ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43317 method = POST path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
38742024-10-17T22:41:59.050ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43317 method = DELETE path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
38752024-10-17T22:41:59.050ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43317 method = GET path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
38762024-10-17T22:41:59.050ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43317 method = GET path = /crucible/0/regions/{id}/snapshots server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
38772024-10-17T22:41:59.050ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43317 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
38782024-10-17T22:41:59.050ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43317 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
38792024-10-17T22:41:59.050ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43317 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
38802024-10-17T22:41:59.050ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43317 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
38812024-10-17T22:41:59.050ZINFOtest_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:43317 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
38822024-10-17T22:41:59.050ZDEBGtest_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:43317 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
38832024-10-17T22:41:59.050ZINFOtest_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:43317 kind = storage server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
38842024-10-17T22:41:59.050ZINFOtest_metrics (nexus): upserting physical disk model = test-model physical_disk_id = 53346b3e-def8-4966-994e-df9db7bd73f4 serial = totally-unique-serial: 53346b3e-def8-4966-994e-df9db7bd73f4 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 vendor = test-vendor
38852024-10-17T22:41:59.051ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
38862024-10-17T22:41:59.051ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
38872024-10-17T22:41:59.051ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
38882024-10-17T22:41:59.051ZTRACtest_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
38892024-10-17T22:41:59.051ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
38902024-10-17T22:41:59.052ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
38912024-10-17T22:41:59.052ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
38922024-10-17T22:41:59.052ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
38932024-10-17T22:41:59.053ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9db53f80-1a0c-4308-a980-c169c6ea2967", "content-length": "167", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
38942024-10-17T22:41:59.053ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0/component/stage0/caboose?firmware_slot=0
38952024-10-17T22:41:59.053ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38962024-10-17T22:41:59.057ZDEBGtest_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
38972024-10-17T22:41:59.059ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b2444611-5921-4b8c-b9d4-382660821e50", "content-length": "175", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
38982024-10-17T22:41:59.060ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0/component/stage0/caboose?firmware_slot=1
38992024-10-17T22:41:59.091ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "83285f5d-c3d1-409d-b563-7c782c3dfc52", "content-length": "175", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
39002024-10-17T22:41:59.093ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0/component/rot/cmpa
39012024-10-17T22:41:59.093ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7df979ae-27ee-419c-8b5a-b532243fe4a9", "content-length": "702", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
39022024-10-17T22:41:59.094ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0/component/rot/cfpa
39032024-10-17T22:41:59.094ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa7d5677-8bf2-4150-a6c2-5a4e8e2c052b", "content-length": "727", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
39042024-10-17T22:41:59.094ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0/component/rot/cfpa
39052024-10-17T22:41:59.095ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "55d1998e-ee64-40c8-8e30-070d15efe986", "content-length": "729", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
39062024-10-17T22:41:59.095ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0/component/rot/cfpa
39072024-10-17T22:41:59.096ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "483e5bf0-e405-451f-9926-6f8c874eab1e", "content-length": "728", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
39082024-10-17T22:41:59.096ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1
39092024-10-17T22:41:59.097ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ba8bb39a-d5f5-4201-949b-328aa6a264e9", "content-length": "734", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
39102024-10-17T22:41:59.097ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1/component/sp/caboose?firmware_slot=0
39112024-10-17T22:41:59.099ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "308fb3b9-6590-47af-b0bd-979000a38b82", "content-length": "111", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
39122024-10-17T22:41:59.100ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1/component/sp/caboose?firmware_slot=1
39132024-10-17T22:41:59.101ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e7a7d7c3-f8a8-48d2-8eac-37a1117b5f92", "content-length": "111", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
39142024-10-17T22:41:59.102ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1/component/rot/caboose?firmware_slot=0
39152024-10-17T22:41:59.103ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "76570162-cab4-4695-9959-16571d8f1231", "content-length": "167", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
39162024-10-17T22:41:59.103ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1/component/rot/caboose?firmware_slot=1
39172024-10-17T22:41:59.107ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "08d12f0c-808f-4baa-90eb-5f3a6d4740e7", "content-length": "167", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
39182024-10-17T22:41:59.107ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1/component/stage0/caboose?firmware_slot=0
39192024-10-17T22:41:59.107ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "69fe6a84-01b0-488c-a40c-3b2190e14084", "content-length": "175", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
39202024-10-17T22:41:59.107ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1/component/stage0/caboose?firmware_slot=1
39212024-10-17T22:41:59.122ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4c2eb22f-8f1b-40a0-a1fe-5c43e8ede3e6", "content-length": "175", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
39222024-10-17T22:41:59.122ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1/component/rot/cmpa
39232024-10-17T22:41:59.122ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8e818b37-8d4a-48c6-afd4-b91d3c1a8e2c", "content-length": "702", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
39242024-10-17T22:41:59.122ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1/component/rot/cfpa
39252024-10-17T22:41:59.122ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "96a491af-d671-4463-bce3-48dc78101de1", "content-length": "727", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
39262024-10-17T22:41:59.122ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1/component/rot/cfpa
39272024-10-17T22:41:59.123ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6d899fe8-8f67-4e73-b56c-228411a484b6", "content-length": "729", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
39282024-10-17T22:41:59.124ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1/component/rot/cfpa
39292024-10-17T22:41:59.124ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ee0f620e-a08b-4cca-b2df-e25000ff0fa4", "content-length": "728", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
39302024-10-17T22:41:59.124ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0
39312024-10-17T22:41:59.124ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "60f6608a-99af-420b-a8e7-383c3b3b763e", "content-length": "723", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
39322024-10-17T22:41:59.124ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0/component/sp/caboose?firmware_slot=0
39332024-10-17T22:41:59.124ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "18d0c3f9-8751-40b9-908d-f3d9593f4d77", "content-length": "109", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
39342024-10-17T22:41:59.124ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0/component/sp/caboose?firmware_slot=1
39352024-10-17T22:41:59.124ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b27fd763-f882-460b-ad3b-624f2000b56e", "content-length": "109", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
39362024-10-17T22:41:59.125ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0/component/rot/caboose?firmware_slot=0
39372024-10-17T22:41:59.125ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3c4a59fd-0d08-4bdd-883b-b1a87e81f477", "content-length": "169", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
39382024-10-17T22:41:59.125ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0/component/rot/caboose?firmware_slot=1
39392024-10-17T22:41:59.125ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bde3a51e-acd2-41d6-b67e-c9648a241394", "content-length": "169", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
39402024-10-17T22:41:59.125ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0/component/stage0/caboose?firmware_slot=0
39412024-10-17T22:41:59.126ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d6948eeb-78cd-45ad-91c2-2fe96b89e2d8", "content-length": "178", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
39422024-10-17T22:41:59.126ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0/component/stage0/caboose?firmware_slot=1
39432024-10-17T22:41:59.127ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4f45c7d8-d90c-418c-befd-ecfc729f8acb", "content-length": "178", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
39442024-10-17T22:41:59.128ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0/component/rot/cmpa
39452024-10-17T22:41:59.128ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d94c6819-a88b-419f-b956-e70fde5d079d", "content-length": "702", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
39462024-10-17T22:41:59.128ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0/component/rot/cfpa
39472024-10-17T22:41:59.129ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "54f3984d-1b17-443a-8631-8a3d76bee3ba", "content-length": "727", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
39482024-10-17T22:41:59.130ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0/component/rot/cfpa
39492024-10-17T22:41:59.130ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a650ebbf-2e75-48af-b1bd-0a0c3125b54c", "content-length": "729", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
39502024-10-17T22:41:59.132ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0/component/rot/cfpa
39512024-10-17T22:41:59.132ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8e024592-816b-4690-a54b-4bcda3e477f2", "content-length": "728", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
39522024-10-17T22:41:59.132ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1
39532024-10-17T22:41:59.132ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0aec027d-d19a-4511-8cf2-63363d4f866d", "content-length": "723", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
39542024-10-17T22:41:59.132ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1/component/sp/caboose?firmware_slot=0
39552024-10-17T22:41:59.134ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6a6469a6-45e3-4849-beb0-98154a3d0c54", "content-length": "109", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
39562024-10-17T22:41:59.134ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1/component/sp/caboose?firmware_slot=1
39572024-10-17T22:41:59.136ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2956ae21-31c5-40c5-961f-093feac36b2a", "content-length": "109", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
39582024-10-17T22:41:59.137ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1/component/rot/caboose?firmware_slot=0
39592024-10-17T22:41:59.138ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "41d4923c-eef4-4983-8729-6a96726c8e92", "content-length": "169", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
39602024-10-17T22:41:59.139ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1/component/rot/caboose?firmware_slot=1
39612024-10-17T22:41:59.139ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0b2bb8d7-2629-45aa-91ee-9e4eba749430", "content-length": "169", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
39622024-10-17T22:41:59.139ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1/component/stage0/caboose?firmware_slot=0
39632024-10-17T22:41:59.141ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d0de480e-77ee-4a26-9934-612b70b9781c", "content-length": "178", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
39642024-10-17T22:41:59.141ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1/component/stage0/caboose?firmware_slot=1
39652024-10-17T22:41:59.143ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d6c2b798-f794-4df8-bdae-49f75e495cb3", "content-length": "178", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
39662024-10-17T22:41:59.144ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1/component/rot/cmpa
39672024-10-17T22:41:59.144ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7de28b4c-47ec-4534-8c5d-25a1c971332d", "content-length": "702", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
39682024-10-17T22:41:59.144ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1/component/rot/cfpa
39692024-10-17T22:41:59.145ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d477f5d4-1d0a-4942-9128-dddf1c09d00a", "content-length": "727", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
39702024-10-17T22:41:59.149ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1/component/rot/cfpa
39712024-10-17T22:41:59.149ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ca5e05bc-b548-4f41-bf0b-b0f290b90732", "content-length": "729", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
39722024-10-17T22:41:59.149ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1/component/rot/cfpa
39732024-10-17T22:41:59.149ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4da380f0-8efe-4404-8859-8b988054823b", "content-length": "728", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
39742024-10-17T22:41:59.149ZDEBGtest_metrics (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:44607
39752024-10-17T22:41:59.150ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:44607 method = GET uri = http://[::1]:44607/ignition
39762024-10-17T22:41:59.150ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:44607 result = Ok(Response { url: "http://[::1]:44607/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "1bd43008-edac-4d53-ac22-d8863514cd6c", "content-length": "835", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
39772024-10-17T22:41:59.150ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:44607 method = GET uri = http://[::1]:44607/sp/switch/0
39782024-10-17T22:41:59.150ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:44607 result = Ok(Response { url: "http://[::1]:44607/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4c0eb789-c5d3-42d3-8895-48e2ee9f96f4", "content-length": "734", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
39792024-10-17T22:41:59.151ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:44607 method = GET uri = http://[::1]:44607/sp/switch/1
39802024-10-17T22:41:59.152ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:44607 result = Ok(Response { url: "http://[::1]:44607/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a1a152a1-519f-4aa7-86bc-e2d650ef29e4", "content-length": "734", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
39812024-10-17T22:41:59.152ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:44607 method = GET uri = http://[::1]:44607/sp/sled/0
39822024-10-17T22:41:59.153ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:44607 result = Ok(Response { url: "http://[::1]:44607/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "688fd956-0961-4eba-ba9f-efb0163e4801", "content-length": "723", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
39832024-10-17T22:41:59.153ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:44607 method = GET uri = http://[::1]:44607/sp/sled/1
39842024-10-17T22:41:59.154ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:44607 result = Ok(Response { url: "http://[::1]:44607/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "39b1ecbd-fd4d-4a8c-af36-5be77aab0e47", "content-length": "723", "date": "Thu, 17 Oct 2024 22:41:58 GMT"} })
39852024-10-17T22:41:59.155ZTRACtest_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
39862024-10-17T22:41:59.155ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
39872024-10-17T22:41:59.155ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
39882024-10-17T22:41:59.155ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
39892024-10-17T22:41:59.156ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
39902024-10-17T22:41:59.169ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39912024-10-17T22:41:59.175ZDEBGtest_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
39922024-10-17T22:41:59.175ZTRACtest_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
39932024-10-17T22:41:59.175ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
39942024-10-17T22:41:59.176ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
39952024-10-17T22:41:59.176ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
39962024-10-17T22:41:59.176ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
39972024-10-17T22:41:59.177ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39982024-10-17T22:41:59.180ZDEBGtest_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
39992024-10-17T22:41:59.180ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
40002024-10-17T22:41:59.181ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
40012024-10-17T22:41:59.181ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
40022024-10-17T22:41:59.215ZDEBGtest_metrics (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:35099
40032024-10-17T22:41:59.216ZDEBGtest_metrics (BackgroundTasks): client request SledAgent = http://[::1]:35099 background_task = inventory_collection body = None method = GET uri = http://[::1]:35099/inventory
40042024-10-17T22:41:59.216ZINFOtest_metrics (nexus): upserting zpool physical_disk_id = 53346b3e-def8-4966-994e-df9db7bd73f4 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 zpool_id = a9f5a864-0a46-482c-b142-74e82a5d2179
40052024-10-17T22:41:59.216ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
40062024-10-17T22:41:59.216ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
40072024-10-17T22:41:59.217ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
40082024-10-17T22:41:59.218ZINFOtest_metrics (dropshot): accepted connection local_addr = [::1]:35099 remote_addr = [::1]:59838 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
40092024-10-17T22:41:59.218ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
40102024-10-17T22:41:59.218ZTRACtest_metrics (dropshot): incoming request local_addr = [::1]:35099 method = GET remote_addr = [::1]:59838 req_id = 45899bbe-64b9-4bec-a737-c756dad67ae5 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /inventory
40112024-10-17T22:41:59.220ZINFOtest_metrics (dropshot): request completed latency_us = 1635 local_addr = [::1]:35099 method = GET remote_addr = [::1]:59838 req_id = 45899bbe-64b9-4bec-a737-c756dad67ae5 response_code = 200 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /inventory
40122024-10-17T22:41:59.221ZTRACtest_metrics (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
40132024-10-17T22:41:59.222ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
40142024-10-17T22:41:59.222ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
40152024-10-17T22:41:59.222ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
40162024-10-17T22:41:59.222ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
40172024-10-17T22:41:59.222ZDEBGtest_metrics (BackgroundTasks): client response SledAgent = http://[::1]:35099 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:35099/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "45899bbe-64b9-4bec-a737-c756dad67ae5", "content-length": "4199", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
40182024-10-17T22:41:59.255ZDEBGtest_metrics (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:40953
40192024-10-17T22:41:59.257ZDEBGtest_metrics (BackgroundTasks): client request SledAgent = http://[::1]:40953 background_task = inventory_collection body = None method = GET uri = http://[::1]:40953/inventory
40202024-10-17T22:41:59.257ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
40212024-10-17T22:41:59.257ZINFOtest_metrics (dropshot): accepted connection local_addr = [::1]:40953 remote_addr = [::1]:57538 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
40222024-10-17T22:41:59.257ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
40232024-10-17T22:41:59.257ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
40242024-10-17T22:41:59.258ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
40252024-10-17T22:41:59.259ZTRACtest_metrics (dropshot): incoming request local_addr = [::1]:40953 method = GET remote_addr = [::1]:57538 req_id = 48c92025-cc03-4d1d-bbf5-7fe6171d15af sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
40262024-10-17T22:41:59.260ZINFOtest_metrics (dropshot): request completed latency_us = 1356 local_addr = [::1]:40953 method = GET remote_addr = [::1]:57538 req_id = 48c92025-cc03-4d1d-bbf5-7fe6171d15af response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
40272024-10-17T22:41:59.262ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40282024-10-17T22:41:59.263ZDEBGtest_metrics (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
40292024-10-17T22:41:59.264ZDEBGtest_metrics (BackgroundTasks): client response SledAgent = http://[::1]:40953 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:40953/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "48c92025-cc03-4d1d-bbf5-7fe6171d15af", "content-length": "7565", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
40302024-10-17T22:41:59.267ZDEBGtest_metrics (BackgroundTasks): finished collection background_task = inventory_collection
40312024-10-17T22:41:59.268ZTRACtest_metrics (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
40322024-10-17T22:41:59.268ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
40332024-10-17T22:41:59.268ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
40342024-10-17T22:41:59.268ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
40352024-10-17T22:41:59.268ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
40362024-10-17T22:41:59.269ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
40372024-10-17T22:41:59.269ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:41:59.219475599","datum":1}\n
40382024-10-17T22:41:59.272ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40392024-10-17T22:41:59.277ZDEBGtest_metrics (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
40402024-10-17T22:41:59.277ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
40412024-10-17T22:41:59.278ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
40422024-10-17T22:41:59.278ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
40432024-10-17T22:41:59.278ZINFOtest_metrics (nexus): upserting dataset address = [::1]:43317 dataset_id = fea564fb-cf80-48da-b821-1172d70693f8 zpool_id = a9f5a864-0a46-482c-b142-74e82a5d2179
40442024-10-17T22:41:59.278ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
40452024-10-17T22:41:59.278ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
40462024-10-17T22:41:59.345ZTRACtest_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
40472024-10-17T22:41:59.345ZTRACtest_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
40482024-10-17T22:41:59.345ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
40492024-10-17T22:41:59.345ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
40502024-10-17T22:41:59.346ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
40512024-10-17T22:41:59.350ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40522024-10-17T22:41:59.355ZDEBGtest_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
40532024-10-17T22:41:59.356ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
40542024-10-17T22:41:59.356ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
40552024-10-17T22:41:59.356ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
40562024-10-17T22:41:59.391ZINFOtest_metrics (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 1e03a7f1-3d84-41a8-bd71-e59407350034
40572024-10-17T22:41:59.391ZDEBGtest_metrics (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 1e03a7f1-3d84-41a8-bd71-e59407350034 time_started = 2024-10-17 22:41:59.004166 UTC
40582024-10-17T22:41:59.391ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 483.409238ms iteration = 7 status = {"collection_id":"1e03a7f1-3d84-41a8-bd71-e59407350034","time_done":"2024-10-17 22:41:59.265267 UTC","time_started":"2024-10-17 22:41:59.004166 UTC"}
40592024-10-17T22:41:59.392ZDEBGtest_metrics (BackgroundTasks): activating background_task = inventory_collection iteration = 8 reason = Signaled
40602024-10-17T22:41:59.392ZTRACtest_metrics (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
40612024-10-17T22:41:59.392ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
40622024-10-17T22:41:59.392ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
40632024-10-17T22:41:59.393ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
40642024-10-17T22:41:59.398ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
40652024-10-17T22:41:59.398ZDEBGtest_metrics (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 5 reason = Dependency
40662024-10-17T22:41:59.398ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 51.782\u{b5}s iteration = 5 status = {"error":"task disabled"}
40672024-10-17T22:41:59.398ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40682024-10-17T22:41:59.399ZDEBGtest_metrics (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
40692024-10-17T22:41:59.399ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
40702024-10-17T22:41:59.399ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
40712024-10-17T22:41:59.400ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
40722024-10-17T22:41:59.400ZTRACtest_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
40732024-10-17T22:41:59.400ZDEBGtest_metrics (InternalApi): roles roles = RoleSet { roles: {} }
40742024-10-17T22:41:59.401ZDEBGtest_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
40752024-10-17T22:41:59.404ZDEBGtest_metrics (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(1b3228a8-106c-4cda-9e3a-742e41b95979, 0), (4cdc8bc0-acad-463c-aa93-04376cc0e9a4, 0), (5feb46a1-e3a3-4c4e-bfbb-6ff6d63eb732, 0), (1e03a7f1-3d84-41a8-bd71-e59407350034, 0)] collection_id = 1b3228a8-106c-4cda-9e3a-742e41b95979
40762024-10-17T22:41:59.405ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
40772024-10-17T22:41:59.405ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
40782024-10-17T22:41:59.405ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
40792024-10-17T22:41:59.445ZDEBGtest_metrics: attempt to fetch latest inventory collection result = Ok("found")
40802024-10-17T22:41:59.501ZINFOtest_metrics (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 1b3228a8-106c-4cda-9e3a-742e41b95979 ncabooses = 24 nclickhouse_keeper_membership = 0 ncollections = 1 ndatasets = 0 nerrors = 0 nnics = 2 nnvme_disk_firmware = 20 nphysical_disks = 20 nrot_pages = 16 nrots = 4 nsled_agent_zones = 2 nsled_agents = 2 nsps = 4 nzones = 6 nzpools = 20
40812024-10-17T22:41:59.501ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
40822024-10-17T22:41:59.501ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
40832024-10-17T22:41:59.501ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
40842024-10-17T22:41:59.518ZDEBGtest_metrics (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(4cdc8bc0-acad-463c-aa93-04376cc0e9a4, 0), (5feb46a1-e3a3-4c4e-bfbb-6ff6d63eb732, 0), (1e03a7f1-3d84-41a8-bd71-e59407350034, 0)]
40852024-10-17T22:41:59.518ZTRACtest_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
40862024-10-17T22:41:59.519ZTRACtest_metrics (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 85 0b 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:59743 req_id = 853e099c-cf0a-495a-b98b-345ef5e591fe
40872024-10-17T22:41:59.519ZDEBGtest_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 34059,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:59743 req_id = 853e099c-cf0a-495a-b98b-345ef5e591fe
40882024-10-17T22:41:59.519ZDEBGtest_metrics (store): query key key = _mgs._tcp
40892024-10-17T22:41:59.521ZDEBGtest_metrics (store): query key key = dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host
40902024-10-17T22:41:59.522ZDEBGtest_metrics (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
40912024-10-17T22:41:59.522ZDEBGtest_metrics (dns): dns response additional_records = [Record { name_labels: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:59743 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 33937, target: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal") })) }, Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 44607, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 853e099c-cf0a-495a-b98b-345ef5e591fe
40922024-10-17T22:41:59.522ZDEBGtest_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 33937, target: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 44607, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2091, tv_nsec: 289213679 } })
40932024-10-17T22:41:59.522ZTRACtest_metrics (DnsResolver): lookup_service_targets: looking up SRV target
40942024-10-17T22:41:59.522ZTRACtest_metrics (DnsResolver): lookup_service_targets: looking up SRV target
40952024-10-17T22:41:59.522ZTRACtest_metrics (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: b1 31 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .1..........-den\n0010: 64 72 69 74 65 2d 30 33 39 62 65 35 36 30 2d 35 drite-039be560-5\n0020: 34 63 63 2d 34 39 65 33 2d 38 38 64 66 2d 31 61 4cc-49e3-88df-1a\n0030: 32 39 64 61 64 62 66 39 31 33 04 68 6f 73 74 0d 29dadbf913.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:58963 req_id = 6d310b3d-cee2-4902-8ce9-607280e2f35b
40962024-10-17T22:41:59.522ZDEBGtest_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 45361,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 48,\n 51,\n 57,\n 98,\n 101,\n 53,\n 54,\n 48,\n 45,\n 53,\n 52,\n 99,\n 99,\n 45,\n 52,\n 57,\n 101,\n 51,\n 45,\n 56,\n 56,\n 100,\n 102,\n 45,\n 49,\n 97,\n 50,\n 57,\n 100,\n 97,\n 100,\n 98,\n 102,\n 57,\n 49,\n 51,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:58963 req_id = 6d310b3d-cee2-4902-8ce9-607280e2f35b
40972024-10-17T22:41:59.522ZDEBGtest_metrics (store): query key key = dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host
40982024-10-17T22:41:59.522ZDEBGtest_metrics (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:58963 query = LowerQuery { name: LowerName(Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 6d310b3d-cee2-4902-8ce9-607280e2f35b
40992024-10-17T22:41:59.522ZTRACtest_metrics (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: ec a4 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:52476 req_id = 2001f6d7-db45-48e7-b8cb-a1d28ebde7bb
41002024-10-17T22:41:59.522ZDEBGtest_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 60580,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52476 req_id = 2001f6d7-db45-48e7-b8cb-a1d28ebde7bb
41012024-10-17T22:41:59.523ZDEBGtest_metrics (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
41022024-10-17T22:41:59.523ZDEBGtest_metrics (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:52476 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 2001f6d7-db45-48e7-b8cb-a1d28ebde7bb
41032024-10-17T22:41:59.597ZTRACtest_metrics (DnsResolver): lookup_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
41042024-10-17T22:41:59.597ZTRACtest_metrics (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 2b 60 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c +`..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:51083 req_id = 94cc7b9e-395f-4bed-8b7f-232d34a12d78
41052024-10-17T22:41:59.598ZDEBGtest_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 11104,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51083 req_id = 94cc7b9e-395f-4bed-8b7f-232d34a12d78
41062024-10-17T22:41:59.598ZDEBGtest_metrics (store): query key key = _clickhouse-admin-keeper._tcp
41072024-10-17T22:41:59.598ZERROtest_metrics (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 11104, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:51083 req_id = 94cc7b9e-395f-4bed-8b7f-232d34a12d78
41082024-10-17T22:41:59.599ZDEBGtest_metrics (BackgroundTasks): begin collection background_task = inventory_collection
41092024-10-17T22:41:59.599ZDEBGtest_metrics (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:33937
41102024-10-17T22:41:59.599ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/ignition
41112024-10-17T22:41:59.601ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "315eae42-ed5c-4c1e-9a79-21395b02d428", "content-length": "835", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
41122024-10-17T22:41:59.602ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0
41132024-10-17T22:41:59.602ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "02ba8367-aeb6-4e77-bf55-367062652173", "content-length": "734", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
41142024-10-17T22:41:59.603ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0/component/sp/caboose?firmware_slot=0
41152024-10-17T22:41:59.604ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "82dc9ef1-c29f-4e25-8922-4107a30df1ea", "content-length": "111", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
41162024-10-17T22:41:59.605ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0/component/sp/caboose?firmware_slot=1
41172024-10-17T22:41:59.606ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "44e605ef-20e8-4419-9cb7-7ddb73ccc910", "content-length": "111", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
41182024-10-17T22:41:59.606ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0/component/rot/caboose?firmware_slot=0
41192024-10-17T22:41:59.608ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f97e732a-c697-47ef-a328-433ab6c50def", "content-length": "167", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
41202024-10-17T22:41:59.608ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0/component/rot/caboose?firmware_slot=1
41212024-10-17T22:41:59.610ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1999768d-9ae8-4b31-a4b7-7106ebc314c5", "content-length": "167", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
41222024-10-17T22:41:59.611ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0/component/stage0/caboose?firmware_slot=0
41232024-10-17T22:41:59.611ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "434dc5c5-b269-4678-b441-fc7cbde1fa13", "content-length": "175", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
41242024-10-17T22:41:59.611ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0/component/stage0/caboose?firmware_slot=1
41252024-10-17T22:41:59.613ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f4357cf4-2f99-4687-bcde-8400ff8916c6", "content-length": "175", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
41262024-10-17T22:41:59.614ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0/component/rot/cmpa
41272024-10-17T22:41:59.614ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3502d9d1-9be4-4ec4-af84-80fac8487e49", "content-length": "702", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
41282024-10-17T22:41:59.614ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0/component/rot/cfpa
41292024-10-17T22:41:59.615ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "bc2d2d55-4fe6-4600-8a73-15834a3bbd1f", "content-length": "727", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
41302024-10-17T22:41:59.616ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0/component/rot/cfpa
41312024-10-17T22:41:59.616ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2268a04c-c3ac-48b5-bd78-d3b8c427ec7e", "content-length": "729", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
41322024-10-17T22:41:59.616ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/0/component/rot/cfpa
41332024-10-17T22:41:59.617ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5ac149e0-d903-4d7e-a859-acbca1155697", "content-length": "728", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
41342024-10-17T22:41:59.617ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1
41352024-10-17T22:41:59.618ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "64984329-fe72-4b45-b4ff-00873af85e88", "content-length": "734", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
41362024-10-17T22:41:59.618ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1/component/sp/caboose?firmware_slot=0
41372024-10-17T22:41:59.620ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "25f2b70c-64f4-4720-b516-adc85f82acfa", "content-length": "111", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
41382024-10-17T22:41:59.621ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1/component/sp/caboose?firmware_slot=1
41392024-10-17T22:41:59.622ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8bcf48e0-99df-4419-8f25-e7b43790c92c", "content-length": "111", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
41402024-10-17T22:41:59.623ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1/component/rot/caboose?firmware_slot=0
41412024-10-17T22:41:59.624ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d7d526bd-df50-4783-8013-3b5313c3bd91", "content-length": "167", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
41422024-10-17T22:41:59.625ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1/component/rot/caboose?firmware_slot=1
41432024-10-17T22:41:59.625ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7e659cd2-8db1-4170-bf4c-2bd4218720a1", "content-length": "167", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
41442024-10-17T22:41:59.627ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1/component/stage0/caboose?firmware_slot=0
41452024-10-17T22:41:59.627ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5f455b13-e29b-413e-9ef9-ad84dbcb60fe", "content-length": "175", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
41462024-10-17T22:41:59.627ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1/component/stage0/caboose?firmware_slot=1
41472024-10-17T22:41:59.629ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c9ba1282-6956-4cce-95dc-75cef06c1838", "content-length": "175", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
41482024-10-17T22:41:59.629ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1/component/rot/cmpa
41492024-10-17T22:41:59.630ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "16b9be6c-a421-49ac-8ec7-2814439f07a3", "content-length": "702", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
41502024-10-17T22:41:59.632ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1/component/rot/cfpa
41512024-10-17T22:41:59.632ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9501a2a8-1ee6-4bcd-af75-8f08c7e7d370", "content-length": "727", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
41522024-10-17T22:41:59.632ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1/component/rot/cfpa
41532024-10-17T22:41:59.632ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "66731db3-5cee-4880-b6ac-f821fe4ff54e", "content-length": "729", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
41542024-10-17T22:41:59.632ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/switch/1/component/rot/cfpa
41552024-10-17T22:41:59.633ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "647fd384-c368-4573-86af-4842cc56a1ef", "content-length": "728", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
41562024-10-17T22:41:59.635ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0
41572024-10-17T22:41:59.635ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "50003723-7f69-4c41-9d04-55a7b1874b65", "content-length": "723", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
41582024-10-17T22:41:59.635ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0/component/sp/caboose?firmware_slot=0
41592024-10-17T22:41:59.636ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa02229e-114f-4b02-8ed7-e0edd3b7cdcd", "content-length": "109", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
41602024-10-17T22:41:59.636ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0/component/sp/caboose?firmware_slot=1
41612024-10-17T22:41:59.637ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9023e1ab-446b-4fef-95c0-c0b72924e0e7", "content-length": "109", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
41622024-10-17T22:41:59.638ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0/component/rot/caboose?firmware_slot=0
41632024-10-17T22:41:59.639ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "56220fe5-8454-45b1-beca-16eee534528f", "content-length": "169", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
41642024-10-17T22:41:59.640ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0/component/rot/caboose?firmware_slot=1
41652024-10-17T22:41:59.641ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4064d933-4653-43c1-ae19-f84e03dd4727", "content-length": "169", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
41662024-10-17T22:41:59.643ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0/component/stage0/caboose?firmware_slot=0
41672024-10-17T22:41:59.643ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "71848b08-fb5f-4c4e-b3b4-ec98d41c1f6c", "content-length": "178", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
41682024-10-17T22:41:59.643ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0/component/stage0/caboose?firmware_slot=1
41692024-10-17T22:41:59.644ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "51268fd5-64e6-444b-96d9-780e9a0494e4", "content-length": "178", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
41702024-10-17T22:41:59.645ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0/component/rot/cmpa
41712024-10-17T22:41:59.645ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c96d7866-5b25-4972-a035-8e433821389a", "content-length": "702", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
41722024-10-17T22:41:59.645ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0/component/rot/cfpa
41732024-10-17T22:41:59.646ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a5c6f9a3-7776-4ac0-8e57-a8ee89d24171", "content-length": "727", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
41742024-10-17T22:41:59.647ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0/component/rot/cfpa
41752024-10-17T22:41:59.647ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1fb90c76-0308-4692-8467-678486de98a7", "content-length": "729", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
41762024-10-17T22:41:59.648ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/0/component/rot/cfpa
41772024-10-17T22:41:59.649ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e599bf99-c395-4f0f-a1ad-a7ae81f24fe9", "content-length": "728", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
41782024-10-17T22:41:59.649ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1
41792024-10-17T22:41:59.650ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6ad216cc-92c3-48fa-ab0a-bfe187ab955d", "content-length": "723", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
41802024-10-17T22:41:59.651ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1/component/sp/caboose?firmware_slot=0
41812024-10-17T22:41:59.652ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "15aba92c-67ac-44ad-8d80-1c1b2292b39b", "content-length": "109", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
41822024-10-17T22:41:59.652ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1/component/sp/caboose?firmware_slot=1
41832024-10-17T22:41:59.654ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "898f0000-d6a2-466a-a68d-e36ae8a41711", "content-length": "109", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
41842024-10-17T22:41:59.655ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1/component/rot/caboose?firmware_slot=0
41852024-10-17T22:41:59.656ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f6617e14-5b9e-4878-8377-8c2b99df7470", "content-length": "169", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
41862024-10-17T22:41:59.656ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1/component/rot/caboose?firmware_slot=1
41872024-10-17T22:41:59.658ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a1c81241-0989-4381-8e57-0176fb7fed7a", "content-length": "169", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
41882024-10-17T22:41:59.659ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1/component/stage0/caboose?firmware_slot=0
41892024-10-17T22:41:59.660ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b54f0d09-d5b6-4323-bc11-ab34739dc885", "content-length": "178", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
41902024-10-17T22:41:59.661ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1/component/stage0/caboose?firmware_slot=1
41912024-10-17T22:41:59.665ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f999126d-7676-461d-a1a3-859316bcbaa6", "content-length": "178", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
41922024-10-17T22:41:59.666ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1/component/rot/cmpa
41932024-10-17T22:41:59.668ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1322b651-1e40-4c3a-a73f-f6bb9ab6824f", "content-length": "702", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
41942024-10-17T22:41:59.668ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1/component/rot/cfpa
41952024-10-17T22:41:59.669ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "889139fa-09f4-47ad-bd4a-f0804c3ab182", "content-length": "727", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
41962024-10-17T22:41:59.669ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1/component/rot/cfpa
41972024-10-17T22:41:59.670ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ae56a950-8d7f-4ce0-81a7-6956ca49de80", "content-length": "729", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
41982024-10-17T22:41:59.671ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:33937 method = GET uri = http://[::1]:33937/sp/sled/1/component/rot/cfpa
41992024-10-17T22:41:59.671ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:33937 result = Ok(Response { url: "http://[::1]:33937/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e0b06030-da65-433d-9491-2367e81c9ff5", "content-length": "728", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
42002024-10-17T22:41:59.671ZDEBGtest_metrics (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:44607
42012024-10-17T22:41:59.671ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:44607 method = GET uri = http://[::1]:44607/ignition
42022024-10-17T22:41:59.673ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:44607 result = Ok(Response { url: "http://[::1]:44607/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "f4e8dbf5-cb89-4617-96e5-cb9982c9847b", "content-length": "835", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
42032024-10-17T22:41:59.673ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:44607 method = GET uri = http://[::1]:44607/sp/switch/0
42042024-10-17T22:41:59.674ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:44607 result = Ok(Response { url: "http://[::1]:44607/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f8135728-940f-48d6-92a6-1dc97a578413", "content-length": "734", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
42052024-10-17T22:41:59.678ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:44607 method = GET uri = http://[::1]:44607/sp/switch/1
42062024-10-17T22:41:59.678ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:44607 result = Ok(Response { url: "http://[::1]:44607/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "57915daa-3052-4859-98d8-c03d08b060fa", "content-length": "734", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
42072024-10-17T22:41:59.678ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:44607 method = GET uri = http://[::1]:44607/sp/sled/0
42082024-10-17T22:41:59.678ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:44607 result = Ok(Response { url: "http://[::1]:44607/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b81b9233-5d6b-4704-bef4-42c8460de640", "content-length": "723", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
42092024-10-17T22:41:59.679ZDEBGtest_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:44607 method = GET uri = http://[::1]:44607/sp/sled/1
42102024-10-17T22:41:59.679ZDEBGtest_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:44607 result = Ok(Response { url: "http://[::1]:44607/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5152496f-14cf-42e4-92d1-bc7b66fa1502", "content-length": "723", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
42112024-10-17T22:41:59.679ZTRACtest_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
42122024-10-17T22:41:59.679ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
42132024-10-17T22:41:59.679ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
42142024-10-17T22:41:59.680ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
42152024-10-17T22:41:59.680ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
42162024-10-17T22:41:59.682ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42172024-10-17T22:41:59.686ZDEBGtest_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
42182024-10-17T22:41:59.687ZTRACtest_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
42192024-10-17T22:41:59.687ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
42202024-10-17T22:41:59.687ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
42212024-10-17T22:41:59.687ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
42222024-10-17T22:41:59.687ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
42232024-10-17T22:41:59.689ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42242024-10-17T22:41:59.697ZDEBGtest_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
42252024-10-17T22:41:59.697ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
42262024-10-17T22:41:59.697ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
42272024-10-17T22:41:59.697ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
42282024-10-17T22:41:59.744ZDEBGtest_metrics (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:35099
42292024-10-17T22:41:59.745ZDEBGtest_metrics (BackgroundTasks): client request SledAgent = http://[::1]:35099 background_task = inventory_collection body = None method = GET uri = http://[::1]:35099/inventory
42302024-10-17T22:41:59.745ZINFOtest_metrics (dropshot): accepted connection local_addr = [::1]:35099 remote_addr = [::1]:59840 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
42312024-10-17T22:41:59.745ZTRACtest_metrics (dropshot): incoming request local_addr = [::1]:35099 method = GET remote_addr = [::1]:59840 req_id = 18e214a6-ea31-4341-a0c3-b49932050dc2 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /inventory
42322024-10-17T22:41:59.747ZINFOtest_metrics (dropshot): request completed latency_us = 446 local_addr = [::1]:35099 method = GET remote_addr = [::1]:59840 req_id = 18e214a6-ea31-4341-a0c3-b49932050dc2 response_code = 200 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /inventory
42332024-10-17T22:41:59.747ZDEBGtest_metrics (BackgroundTasks): client response SledAgent = http://[::1]:35099 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:35099/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "18e214a6-ea31-4341-a0c3-b49932050dc2", "content-length": "4199", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
42342024-10-17T22:41:59.779ZDEBGtest_metrics (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:40953
42352024-10-17T22:41:59.780ZDEBGtest_metrics (BackgroundTasks): client request SledAgent = http://[::1]:40953 background_task = inventory_collection body = None method = GET uri = http://[::1]:40953/inventory
42362024-10-17T22:41:59.780ZINFOtest_metrics (dropshot): accepted connection local_addr = [::1]:40953 remote_addr = [::1]:57540 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
42372024-10-17T22:41:59.780ZTRACtest_metrics (dropshot): incoming request local_addr = [::1]:40953 method = GET remote_addr = [::1]:57540 req_id = cc6e4e66-c005-4f42-b44e-da20d6e95504 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
42382024-10-17T22:41:59.780ZINFOtest_metrics (dropshot): request completed latency_us = 527 local_addr = [::1]:40953 method = GET remote_addr = [::1]:57540 req_id = cc6e4e66-c005-4f42-b44e-da20d6e95504 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
42392024-10-17T22:41:59.782ZDEBGtest_metrics (BackgroundTasks): client response SledAgent = http://[::1]:40953 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:40953/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc6e4e66-c005-4f42-b44e-da20d6e95504", "content-length": "7565", "date": "Thu, 17 Oct 2024 22:41:59 GMT"} })
42402024-10-17T22:41:59.783ZDEBGtest_metrics (BackgroundTasks): finished collection background_task = inventory_collection
42412024-10-17T22:41:59.784ZTRACtest_metrics (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
42422024-10-17T22:41:59.785ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
42432024-10-17T22:41:59.785ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
42442024-10-17T22:41:59.785ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
42452024-10-17T22:41:59.786ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
42462024-10-17T22:41:59.791ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42472024-10-17T22:41:59.793ZDEBGtest_metrics (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
42482024-10-17T22:41:59.793ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
42492024-10-17T22:41:59.794ZDEBGtest_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
42502024-10-17T22:41:59.794ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
42512024-10-17T22:42:00.049ZINFOtest_metrics (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = b64fdb21-4736-43e0-9da7-8704bd5135dc
42522024-10-17T22:42:00.050ZDEBGtest_metrics (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = b64fdb21-4736-43e0-9da7-8704bd5135dc time_started = 2024-10-17 22:41:59.599137 UTC
42532024-10-17T22:42:00.050ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 657.842193ms iteration = 8 status = {"collection_id":"b64fdb21-4736-43e0-9da7-8704bd5135dc","time_done":"2024-10-17 22:41:59.782876 UTC","time_started":"2024-10-17 22:41:59.599137 UTC"}
42542024-10-17T22:42:00.050ZDEBGtest_metrics (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 6 reason = Dependency
42552024-10-17T22:42:00.050ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 87.222\u{b5}s iteration = 6 status = {"error":"task disabled"}
42562024-10-17T22:42:00.186ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
42572024-10-17T22:42:00.189ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
42582024-10-17T22:42:00.243ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
42592024-10-17T22:42:00.244ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
42602024-10-17T22:42:00.245ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
42612024-10-17T22:42:00.250ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
42622024-10-17T22:42:00.251ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:00.187163934","datum":2}\n
42632024-10-17T22:42:00.255ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
42642024-10-17T22:42:00.256ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
42652024-10-17T22:42:01.186ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
42662024-10-17T22:42:01.190ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
42672024-10-17T22:42:01.244ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
42682024-10-17T22:42:01.250ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
42692024-10-17T22:42:01.250ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
42702024-10-17T22:42:01.252ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
42712024-10-17T22:42:01.252ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:01.187443072","datum":3}\n
42722024-10-17T22:42:01.258ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
42732024-10-17T22:42:01.258ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
42742024-10-17T22:42:02.187ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
42752024-10-17T22:42:02.194ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
42762024-10-17T22:42:02.245ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
42772024-10-17T22:42:02.246ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
42782024-10-17T22:42:02.246ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
42792024-10-17T22:42:02.251ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
42802024-10-17T22:42:02.251ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:02.188124899","datum":4}\n
42812024-10-17T22:42:02.273ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
42822024-10-17T22:42:02.274ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
42832024-10-17T22:42:03.187ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
42842024-10-17T22:42:03.192ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
42852024-10-17T22:42:03.245ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
42862024-10-17T22:42:03.245ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
42872024-10-17T22:42:03.245ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
42882024-10-17T22:42:03.249ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
42892024-10-17T22:42:03.251ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:03.187663759","datum":5}\n
42902024-10-17T22:42:03.256ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
42912024-10-17T22:42:03.257ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
42922024-10-17T22:42:04.186ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
42932024-10-17T22:42:04.189ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
42942024-10-17T22:42:04.244ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
42952024-10-17T22:42:04.244ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
42962024-10-17T22:42:04.248ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
42972024-10-17T22:42:04.262ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
42982024-10-17T22:42:04.262ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:04.187498036","datum":6}\n
42992024-10-17T22:42:04.268ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
43002024-10-17T22:42:04.268ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
43012024-10-17T22:42:05.186ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
43022024-10-17T22:42:05.189ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
43032024-10-17T22:42:05.243ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
43042024-10-17T22:42:05.243ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
43052024-10-17T22:42:05.244ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
43062024-10-17T22:42:05.249ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
43072024-10-17T22:42:05.249ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:05.187233520","datum":7}\n
43082024-10-17T22:42:05.264ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
43092024-10-17T22:42:05.272ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
43102024-10-17T22:42:06.186ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
43112024-10-17T22:42:06.191ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
43122024-10-17T22:42:06.243ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
43132024-10-17T22:42:06.244ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
43142024-10-17T22:42:06.245ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
43152024-10-17T22:42:06.250ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
43162024-10-17T22:42:06.250ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:06.187393495","datum":8}\n
43172024-10-17T22:42:06.282ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
43182024-10-17T22:42:06.282ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
43192024-10-17T22:42:07.186ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
43202024-10-17T22:42:07.189ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
43212024-10-17T22:42:07.243ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
43222024-10-17T22:42:07.243ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
43232024-10-17T22:42:07.244ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
43242024-10-17T22:42:07.251ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
43252024-10-17T22:42:07.252ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:07.186944995","datum":9}\n
43262024-10-17T22:42:07.258ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
43272024-10-17T22:42:07.258ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
43282024-10-17T22:42:08.187ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
43292024-10-17T22:42:08.190ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
43302024-10-17T22:42:08.244ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
43312024-10-17T22:42:08.244ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
43322024-10-17T22:42:08.244ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
43332024-10-17T22:42:08.249ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
43342024-10-17T22:42:08.250ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:08.188036022","datum":10}\n
43352024-10-17T22:42:08.257ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
43362024-10-17T22:42:08.259ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
43372024-10-17T22:42:09.187ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
43382024-10-17T22:42:09.190ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
43392024-10-17T22:42:09.244ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
43402024-10-17T22:42:09.244ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
43412024-10-17T22:42:09.245ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
43422024-10-17T22:42:09.249ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
43432024-10-17T22:42:09.250ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:09.187713536","datum":11}\n
43442024-10-17T22:42:09.254ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
43452024-10-17T22:42:09.254ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
43462024-10-17T22:42:10.187ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
43472024-10-17T22:42:10.190ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
43482024-10-17T22:42:10.244ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
43492024-10-17T22:42:10.244ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
43502024-10-17T22:42:10.245ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
43512024-10-17T22:42:10.253ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
43522024-10-17T22:42:10.255ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:10.187622295","datum":12}\n
43532024-10-17T22:42:10.259ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
43542024-10-17T22:42:10.259ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
43552024-10-17T22:42:11.189ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
43562024-10-17T22:42:11.189ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
43572024-10-17T22:42:11.244ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
43582024-10-17T22:42:11.244ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
43592024-10-17T22:42:11.245ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
43602024-10-17T22:42:11.250ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
43612024-10-17T22:42:11.250ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:11.187497943","datum":13}\n
43622024-10-17T22:42:11.256ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
43632024-10-17T22:42:11.256ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
43642024-10-17T22:42:12.186ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
43652024-10-17T22:42:12.189ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
43662024-10-17T22:42:12.245ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
43672024-10-17T22:42:12.245ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
43682024-10-17T22:42:12.245ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
43692024-10-17T22:42:12.252ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
43702024-10-17T22:42:12.256ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:12.187182126","datum":14}\n
43712024-10-17T22:42:12.262ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
43722024-10-17T22:42:12.262ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
43732024-10-17T22:42:13.186ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
43742024-10-17T22:42:13.189ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
43752024-10-17T22:42:13.243ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
43762024-10-17T22:42:13.244ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
43772024-10-17T22:42:13.245ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
43782024-10-17T22:42:13.250ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
43792024-10-17T22:42:13.250ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:13.187057884","datum":15}\n
43802024-10-17T22:42:13.255ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
43812024-10-17T22:42:13.255ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
43822024-10-17T22:42:14.186ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
43832024-10-17T22:42:14.189ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
43842024-10-17T22:42:14.244ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
43852024-10-17T22:42:14.244ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
43862024-10-17T22:42:14.245ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
43872024-10-17T22:42:14.254ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
43882024-10-17T22:42:14.254ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:14.187310958","datum":16}\n
43892024-10-17T22:42:14.261ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
43902024-10-17T22:42:14.261ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
43912024-10-17T22:42:15.190ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
43922024-10-17T22:42:15.190ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
43932024-10-17T22:42:15.246ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
43942024-10-17T22:42:15.246ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
43952024-10-17T22:42:15.246ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
43962024-10-17T22:42:15.252ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
43972024-10-17T22:42:15.253ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:15.187876852","datum":17}\n
43982024-10-17T22:42:15.257ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
43992024-10-17T22:42:15.257ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
44002024-10-17T22:42:16.186ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
44012024-10-17T22:42:16.189ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
44022024-10-17T22:42:16.245ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
44032024-10-17T22:42:16.247ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
44042024-10-17T22:42:16.247ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
44052024-10-17T22:42:16.252ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
44062024-10-17T22:42:16.253ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:16.187254007","datum":18}\n
44072024-10-17T22:42:16.261ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
44082024-10-17T22:42:16.261ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
44092024-10-17T22:42:17.187ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
44102024-10-17T22:42:17.195ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
44112024-10-17T22:42:17.243ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
44122024-10-17T22:42:17.244ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
44132024-10-17T22:42:17.245ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
44142024-10-17T22:42:17.251ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
44152024-10-17T22:42:17.251ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:17.188456267","datum":19}\n
44162024-10-17T22:42:17.256ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
44172024-10-17T22:42:17.257ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
44182024-10-17T22:42:18.186ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
44192024-10-17T22:42:18.189ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
44202024-10-17T22:42:18.244ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
44212024-10-17T22:42:18.244ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
44222024-10-17T22:42:18.245ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
44232024-10-17T22:42:18.250ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
44242024-10-17T22:42:18.250ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:18.187304080","datum":20}\n
44252024-10-17T22:42:18.255ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
44262024-10-17T22:42:18.255ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
44272024-10-17T22:42:19.186ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
44282024-10-17T22:42:19.190ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
44292024-10-17T22:42:19.244ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
44302024-10-17T22:42:19.244ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
44312024-10-17T22:42:19.246ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
44322024-10-17T22:42:19.247ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
44332024-10-17T22:42:19.248ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:19.187316481","datum":21}\n
44342024-10-17T22:42:19.254ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
44352024-10-17T22:42:19.256ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
44362024-10-17T22:42:19.279ZDEBGtest_metrics (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 2 reason = Timeout
44372024-10-17T22:42:19.279ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database
44382024-10-17T22:42:19.279ZDEBGtest_metrics (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
44392024-10-17T22:42:19.280ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
44402024-10-17T22:42:19.280ZDEBGtest_metrics (BackgroundTasks): activating background_task = bfd_manager iteration = 2 reason = Timeout
44412024-10-17T22:42:19.280ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database
44422024-10-17T22:42:19.280ZDEBGtest_metrics (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
44432024-10-17T22:42:19.281ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
44442024-10-17T22:42:19.281ZDEBGtest_metrics (BackgroundTasks): activating background_task = phantom_disks iteration = 2 reason = Timeout
44452024-10-17T22:42:19.284ZTRACtest_metrics (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
44462024-10-17T22:42:19.284ZTRACtest_metrics (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 6f ee 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 o............_de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:63368 req_id = 9e3c02b7-6698-4279-8fdd-35a3cde03413
44472024-10-17T22:42:19.284ZDEBGtest_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 28654,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63368 req_id = 9e3c02b7-6698-4279-8fdd-35a3cde03413
44482024-10-17T22:42:19.285ZDEBGtest_metrics (store): query key key = _dendrite._tcp
44492024-10-17T22:42:19.285ZDEBGtest_metrics (store): query key key = dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host
44502024-10-17T22:42:19.285ZDEBGtest_metrics (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
44512024-10-17T22:42:19.285ZDEBGtest_metrics (dns): dns response additional_records = [Record { name_labels: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:63368 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 36237, target: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal") })) }, Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 36313, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 9e3c02b7-6698-4279-8fdd-35a3cde03413
44522024-10-17T22:42:19.285ZTRACtest_metrics (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
44532024-10-17T22:42:19.287ZDEBGtest_metrics (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 36237, target: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 36313, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2111, tv_nsec: 56236472 } })
44542024-10-17T22:42:19.288ZTRACtest_metrics (DnsResolver): lookup_service_targets: looking up SRV target
44552024-10-17T22:42:19.288ZTRACtest_metrics (DnsResolver): lookup_service_targets: looking up SRV target
44562024-10-17T22:42:19.288ZDEBGtest_metrics (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 7 reason = Timeout
44572024-10-17T22:42:19.288ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 37.881\u{b5}s iteration = 7 status = {"error":"task disabled"}
44582024-10-17T22:42:19.289ZDEBGtest_metrics (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 2 reason = Timeout
44592024-10-17T22:42:19.289ZTRACtest_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet
44602024-10-17T22:42:19.289ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = switch_port_config_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
44612024-10-17T22:42:19.289ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database
44622024-10-17T22:42:19.289ZDEBGtest_metrics (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
44632024-10-17T22:42:19.289ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
44642024-10-17T22:42:19.289ZDEBGtest_metrics (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 2 reason = Timeout
44652024-10-17T22:42:19.289ZTRACtest_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory
44662024-10-17T22:42:19.290ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_zone_nat_tracker resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
44672024-10-17T22:42:19.290ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database
44682024-10-17T22:42:19.290ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
44692024-10-17T22:42:19.290ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
44702024-10-17T22:42:19.290ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 8.33843ms iteration = 2 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
44712024-10-17T22:42:19.290ZDEBGtest_metrics (BackgroundTasks): activating background_task = v2p_manager iteration = 2 reason = Timeout
44722024-10-17T22:42:19.290ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
44732024-10-17T22:42:19.290ZDEBGtest_metrics (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
44742024-10-17T22:42:19.290ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
44752024-10-17T22:42:19.291ZTRACtest_metrics (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 7b bd 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 {............_de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:63105 req_id = df211ca2-e65c-43de-bbfb-973ad11dd177
44762024-10-17T22:42:19.291ZDEBGtest_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 31677,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63105 req_id = df211ca2-e65c-43de-bbfb-973ad11dd177
44772024-10-17T22:42:19.291ZDEBGtest_metrics (store): query key key = _dendrite._tcp
44782024-10-17T22:42:19.291ZDEBGtest_metrics (store): query key key = dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host
44792024-10-17T22:42:19.291ZDEBGtest_metrics (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
44802024-10-17T22:42:19.291ZDEBGtest_metrics (dns): dns response additional_records = [Record { name_labels: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:63105 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 36237, target: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal") })) }, Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 36313, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = df211ca2-e65c-43de-bbfb-973ad11dd177
44812024-10-17T22:42:19.292ZDEBGtest_metrics (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 36237, target: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 36313, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2111, tv_nsec: 61684123 } })
44822024-10-17T22:42:19.293ZTRACtest_metrics (DnsResolver): lookup_service_targets: looking up SRV target
44832024-10-17T22:42:19.293ZTRACtest_metrics (DnsResolver): lookup_service_targets: looking up SRV target
44842024-10-17T22:42:19.293ZDEBGtest_metrics (BackgroundTasks): activating background_task = instance_watcher iteration = 2 reason = Timeout
44852024-10-17T22:42:19.293ZTRACtest_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet
44862024-10-17T22:42:19.293ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = instance_watcher resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
44872024-10-17T22:42:19.293ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database
44882024-10-17T22:42:19.293ZDEBGtest_metrics (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
44892024-10-17T22:42:19.294ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
44902024-10-17T22:42:19.294ZTRACtest_metrics (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 26 2c 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e &,..........-den\n0010: 64 72 69 74 65 2d 30 33 39 62 65 35 36 30 2d 35 drite-039be560-5\n0020: 34 63 63 2d 34 39 65 33 2d 38 38 64 66 2d 31 61 4cc-49e3-88df-1a\n0030: 32 39 64 61 64 62 66 39 31 33 04 68 6f 73 74 0d 29dadbf913.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:64080 req_id = 83d52684-6ed5-4c97-a8ce-f2a6bdf9006d
44912024-10-17T22:42:19.294ZDEBGtest_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 9772,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 48,\n 51,\n 57,\n 98,\n 101,\n 53,\n 54,\n 48,\n 45,\n 53,\n 52,\n 99,\n 99,\n 45,\n 52,\n 57,\n 101,\n 51,\n 45,\n 56,\n 56,\n 100,\n 102,\n 45,\n 49,\n 97,\n 50,\n 57,\n 100,\n 97,\n 100,\n 98,\n 102,\n 57,\n 49,\n 51,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64080 req_id = 83d52684-6ed5-4c97-a8ce-f2a6bdf9006d
44922024-10-17T22:42:19.294ZDEBGtest_metrics (store): query key key = dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host
44932024-10-17T22:42:19.294ZDEBGtest_metrics (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:64080 query = LowerQuery { name: LowerName(Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 83d52684-6ed5-4c97-a8ce-f2a6bdf9006d
44942024-10-17T22:42:19.294ZTRACtest_metrics (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 88 88 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:54681 req_id = 3f77f9ce-1c8e-4a03-ba38-9b0cee946b8e
44952024-10-17T22:42:19.294ZDEBGtest_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 34952,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54681 req_id = 3f77f9ce-1c8e-4a03-ba38-9b0cee946b8e
44962024-10-17T22:42:19.294ZDEBGtest_metrics (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
44972024-10-17T22:42:19.295ZDEBGtest_metrics (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:54681 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 3f77f9ce-1c8e-4a03-ba38-9b0cee946b8e
44982024-10-17T22:42:19.295ZDEBGtest_metrics (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44992024-10-17T22:42:19.300ZDEBGtest_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
45002024-10-17T22:42:19.300ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database
45012024-10-17T22:42:19.301ZDEBGtest_metrics (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
45022024-10-17T22:42:19.301ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
45032024-10-17T22:42:19.302ZINFOtest_metrics (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector
45042024-10-17T22:42:19.343ZINFOtest_metrics (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector zone_address = ::1
45052024-10-17T22:42:19.343ZDEBGtest_metrics (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[::1]:12225/local/switch-id
45062024-10-17T22:42:19.343ZDEBGtest_metrics (BackgroundTasks): activating background_task = instance_updater iteration = 2 reason = Timeout
45072024-10-17T22:42:19.344ZINFOtest_metrics (BackgroundTasks): background instance updater explicitly disabled background_task = instance_updater
45082024-10-17T22:42:19.344ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = instance_updater elapsed = 183.434\u{b5}s iteration = 2 status = {"destroyed_active_vmms":0,"disabled":true,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
45092024-10-17T22:42:19.344ZDEBGtest_metrics (BackgroundTasks): activating background_task = vpc_route_manager iteration = 2 reason = Timeout
45102024-10-17T22:42:19.344ZINFOtest_metrics (BackgroundTasks): VPC route manager running background_task = vpc_route_manager
45112024-10-17T22:42:19.344ZTRACtest_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet
45122024-10-17T22:42:19.344ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = vpc_route_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
45132024-10-17T22:42:19.345ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
45142024-10-17T22:42:19.345ZDEBGtest_metrics (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
45152024-10-17T22:42:19.345ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
45162024-10-17T22:42:19.346ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45172024-10-17T22:42:19.355ZDEBGtest_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
45182024-10-17T22:42:19.355ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database
45192024-10-17T22:42:19.355ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
45202024-10-17T22:42:19.356ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
45212024-10-17T22:42:19.357ZDEBGtest_metrics (MgsClient): client response background_task = nat_v4_garbage_collector result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
45222024-10-17T22:42:19.357ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = nat_v4_garbage_collector reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
45232024-10-17T22:42:19.357ZTRACtest_metrics (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 79 5d 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e y]..........-den\n0010: 64 72 69 74 65 2d 30 33 39 62 65 35 36 30 2d 35 drite-039be560-5\n0020: 34 63 63 2d 34 39 65 33 2d 38 38 64 66 2d 31 61 4cc-49e3-88df-1a\n0030: 32 39 64 61 64 62 66 39 31 33 04 68 6f 73 74 0d 29dadbf913.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:63026 req_id = c3bf3f7a-cccb-4a12-a083-9eff9b246737
45242024-10-17T22:42:19.357ZDEBGtest_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 31069,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 48,\n 51,\n 57,\n 98,\n 101,\n 53,\n 54,\n 48,\n 45,\n 53,\n 52,\n 99,\n 99,\n 45,\n 52,\n 57,\n 101,\n 51,\n 45,\n 56,\n 56,\n 100,\n 102,\n 45,\n 49,\n 97,\n 50,\n 57,\n 100,\n 97,\n 100,\n 98,\n 102,\n 57,\n 49,\n 51,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63026 req_id = c3bf3f7a-cccb-4a12-a083-9eff9b246737
45252024-10-17T22:42:19.357ZDEBGtest_metrics (store): query key key = dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host
45262024-10-17T22:42:19.357ZDEBGtest_metrics (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:63026 query = LowerQuery { name: LowerName(Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = c3bf3f7a-cccb-4a12-a083-9eff9b246737
45272024-10-17T22:42:19.358ZTRACtest_metrics (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 1b d2 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:54830 req_id = 85fe5573-c334-4596-bded-ac771fa1d91a
45282024-10-17T22:42:19.358ZDEBGtest_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 7122,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54830 req_id = 85fe5573-c334-4596-bded-ac771fa1d91a
45292024-10-17T22:42:19.360ZDEBGtest_metrics (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
45302024-10-17T22:42:19.361ZDEBGtest_metrics (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:54830 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 85fe5573-c334-4596-bded-ac771fa1d91a
45312024-10-17T22:42:19.361ZDEBGtest_metrics (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45322024-10-17T22:42:19.365ZDEBGtest_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
45332024-10-17T22:42:19.365ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database
45342024-10-17T22:42:19.365ZDEBGtest_metrics (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
45352024-10-17T22:42:19.365ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
45362024-10-17T22:42:19.365ZTRACtest_metrics (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
45372024-10-17T22:42:19.366ZINFOtest_metrics (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager
45382024-10-17T22:42:19.406ZINFOtest_metrics (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager zone_address = ::1
45392024-10-17T22:42:19.410ZDEBGtest_metrics (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[::1]:12225/local/switch-id
45402024-10-17T22:42:19.411ZDEBGtest_metrics (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45412024-10-17T22:42:19.412ZDEBGtest_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
45422024-10-17T22:42:19.412ZTRACtest_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet
45432024-10-17T22:42:19.412ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = vpc_route_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
45442024-10-17T22:42:19.413ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
45452024-10-17T22:42:19.413ZDEBGtest_metrics (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
45462024-10-17T22:42:19.413ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
45472024-10-17T22:42:19.414ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database
45482024-10-17T22:42:19.414ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
45492024-10-17T22:42:19.415ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
45502024-10-17T22:42:19.420ZDEBGtest_metrics (MgsClient): client response background_task = bfd_manager result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
45512024-10-17T22:42:19.421ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = bfd_manager reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
45522024-10-17T22:42:19.423ZTRACtest_metrics (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 91 98 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 ............._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:61369 req_id = 1a476ad9-5dc2-45d3-a795-45802bfb322b
45532024-10-17T22:42:19.423ZDEBGtest_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 37272,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:61369 req_id = 1a476ad9-5dc2-45d3-a795-45802bfb322b
45542024-10-17T22:42:19.424ZDEBGtest_metrics (store): query key key = _dendrite._tcp
45552024-10-17T22:42:19.424ZDEBGtest_metrics (store): query key key = dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host
45562024-10-17T22:42:19.424ZDEBGtest_metrics (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
45572024-10-17T22:42:19.424ZDEBGtest_metrics (dns): dns response additional_records = [Record { name_labels: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:61369 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 36237, target: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal") })) }, Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 36313, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 1a476ad9-5dc2-45d3-a795-45802bfb322b
45582024-10-17T22:42:19.425ZDEBGtest_metrics (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45592024-10-17T22:42:19.434ZDEBGtest_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
45602024-10-17T22:42:19.434ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
45612024-10-17T22:42:19.439ZDEBGtest_metrics (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
45622024-10-17T22:42:19.440ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
45632024-10-17T22:42:19.441ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
45642024-10-17T22:42:19.441ZDEBGtest_metrics (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
45652024-10-17T22:42:19.442ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
45662024-10-17T22:42:19.444ZDEBGtest_metrics (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 36237, target: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 36313, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2111, tv_nsec: 213359428 } })
45672024-10-17T22:42:19.448ZTRACtest_metrics (DnsResolver): lookup_service_targets: looking up SRV target
45682024-10-17T22:42:19.448ZTRACtest_metrics (DnsResolver): lookup_service_targets: looking up SRV target
45692024-10-17T22:42:19.570ZINFOtest_metrics (BackgroundTasks): VPC route manager sled 039be560-54cc-49e3-88df-1a29dadbf913 background_task = vpc_route_manager
45702024-10-17T22:42:19.570ZDEBGtest_metrics (BackgroundTasks): client request SledAgent = 039be560-54cc-49e3-88df-1a29dadbf913 background_task = vpc_route_manager body = None method = GET uri = http://[::1]:35099/vpc-routes
45712024-10-17T22:42:19.574ZTRACtest_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet
45722024-10-17T22:42:19.576ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = v2p_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
45732024-10-17T22:42:19.576ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
45742024-10-17T22:42:19.576ZDEBGtest_metrics (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
45752024-10-17T22:42:19.577ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
45762024-10-17T22:42:19.577ZINFOtest_metrics (BackgroundTasks): all instance checks complete background_task = instance_watcher pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
45772024-10-17T22:42:19.577ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 280.501643ms iteration = 2 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
45782024-10-17T22:42:19.577ZDEBGtest_metrics (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45792024-10-17T22:42:19.580ZDEBGtest_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
45802024-10-17T22:42:19.581ZTRACtest_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet
45812024-10-17T22:42:19.581ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = v2p_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
45822024-10-17T22:42:19.581ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
45832024-10-17T22:42:19.581ZDEBGtest_metrics (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
45842024-10-17T22:42:19.581ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
45852024-10-17T22:42:19.581ZINFOtest_metrics (dropshot): accepted connection local_addr = [::1]:35099 remote_addr = [::1]:59842 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
45862024-10-17T22:42:19.582ZTRACtest_metrics (dropshot): incoming request local_addr = [::1]:35099 method = GET remote_addr = [::1]:59842 req_id = 65a3e27f-5094-4c31-ac32-d82979005bc0 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /vpc-routes
45872024-10-17T22:42:19.586ZTRACtest_metrics (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: ac 2a 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .*..........-den\n0010: 64 72 69 74 65 2d 30 33 39 62 65 35 36 30 2d 35 drite-039be560-5\n0020: 34 63 63 2d 34 39 65 33 2d 38 38 64 66 2d 31 61 4cc-49e3-88df-1a\n0030: 32 39 64 61 64 62 66 39 31 33 04 68 6f 73 74 0d 29dadbf913.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:64013 req_id = 6890430f-b885-43a5-b483-4b7d7e72f528
45882024-10-17T22:42:19.586ZDEBGtest_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 44074,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 48,\n 51,\n 57,\n 98,\n 101,\n 53,\n 54,\n 48,\n 45,\n 53,\n 52,\n 99,\n 99,\n 45,\n 52,\n 57,\n 101,\n 51,\n 45,\n 56,\n 56,\n 100,\n 102,\n 45,\n 49,\n 97,\n 50,\n 57,\n 100,\n 97,\n 100,\n 98,\n 102,\n 57,\n 49,\n 51,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64013 req_id = 6890430f-b885-43a5-b483-4b7d7e72f528
45892024-10-17T22:42:19.586ZDEBGtest_metrics (store): query key key = dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host
45902024-10-17T22:42:19.587ZDEBGtest_metrics (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:64013 query = LowerQuery { name: LowerName(Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 6890430f-b885-43a5-b483-4b7d7e72f528
45912024-10-17T22:42:19.587ZTRACtest_metrics (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 00 cf 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:62757 req_id = d33f8105-947e-427a-bca7-9c1fdbe5e994
45922024-10-17T22:42:19.587ZDEBGtest_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 207,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62757 req_id = d33f8105-947e-427a-bca7-9c1fdbe5e994
45932024-10-17T22:42:19.587ZDEBGtest_metrics (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
45942024-10-17T22:42:19.587ZDEBGtest_metrics (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:62757 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = d33f8105-947e-427a-bca7-9c1fdbe5e994
45952024-10-17T22:42:19.587ZINFOtest_metrics (dropshot): request completed latency_us = 1237 local_addr = [::1]:35099 method = GET remote_addr = [::1]:59842 req_id = 65a3e27f-5094-4c31-ac32-d82979005bc0 response_code = 200 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /vpc-routes
45962024-10-17T22:42:19.587ZDEBGtest_metrics (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45972024-10-17T22:42:19.588ZDEBGtest_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
45982024-10-17T22:42:19.589ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
45992024-10-17T22:42:19.589ZDEBGtest_metrics (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
46002024-10-17T22:42:19.589ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
46012024-10-17T22:42:19.589ZINFOtest_metrics (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc
46022024-10-17T22:42:19.631ZINFOtest_metrics (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc zone_address = ::1
46032024-10-17T22:42:19.633ZDEBGtest_metrics (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc uri = http://[::1]:12225/local/switch-id
46042024-10-17T22:42:19.634ZDEBGtest_metrics (BackgroundTasks): client response SledAgent = 039be560-54cc-49e3-88df-1a29dadbf913 background_task = vpc_route_manager result = Ok(Response { url: "http://[::1]:35099/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "65a3e27f-5094-4c31-ac32-d82979005bc0", "content-length": "2", "date": "Thu, 17 Oct 2024 22:42:19 GMT"} })
46052024-10-17T22:42:19.634ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
46062024-10-17T22:42:19.635ZDEBGtest_metrics (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
46072024-10-17T22:42:19.635ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
46082024-10-17T22:42:19.635ZDEBGtest_metrics (MgsClient): client response background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
46092024-10-17T22:42:19.635ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
46102024-10-17T22:42:19.711ZDEBGtest_metrics (BackgroundTasks): client request SledAgent = 039be560-54cc-49e3-88df-1a29dadbf913 background_task = v2p_manager body = None method = GET uri = http://[::1]:35099/v2p
46112024-10-17T22:42:19.717ZINFOtest_metrics (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager sled = sim-039be560
46122024-10-17T22:42:19.717ZDEBGtest_metrics (BackgroundTasks): client request SledAgent = 039be560-54cc-49e3-88df-1a29dadbf913 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[::1]:35099/eip-gateways
46132024-10-17T22:42:19.718ZINFOtest_metrics (dropshot): accepted connection local_addr = [::1]:35099 remote_addr = [::1]:59844 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
46142024-10-17T22:42:19.719ZTRACtest_metrics (dropshot): incoming request local_addr = [::1]:35099 method = PUT remote_addr = [::1]:59842 req_id = 807432ec-6cd2-4140-a2cc-89298baa78e1 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /eip-gateways
46152024-10-17T22:42:19.720ZINFOtest_metrics (dropshot): request completed latency_us = 326 local_addr = [::1]:35099 method = PUT remote_addr = [::1]:59842 req_id = 807432ec-6cd2-4140-a2cc-89298baa78e1 response_code = 204 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /eip-gateways
46162024-10-17T22:42:19.721ZTRACtest_metrics (dropshot): incoming request local_addr = [::1]:35099 method = GET remote_addr = [::1]:59844 req_id = bba0f5dd-27bb-47e1-995a-4bb7d9992fc8 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /v2p
46172024-10-17T22:42:19.721ZDEBGtest_metrics (BackgroundTasks): client response SledAgent = 039be560-54cc-49e3-88df-1a29dadbf913 background_task = vpc_route_manager result = Ok(Response { url: "http://[::1]:35099/eip-gateways", status: 204, headers: {"x-request-id": "807432ec-6cd2-4140-a2cc-89298baa78e1", "date": "Thu, 17 Oct 2024 22:42:19 GMT"} })
46182024-10-17T22:42:19.722ZINFOtest_metrics (BackgroundTasks): VPC route manager sled b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager
46192024-10-17T22:42:19.723ZDEBGtest_metrics (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager body = None method = GET uri = http://[::1]:40953/vpc-routes
46202024-10-17T22:42:19.723ZINFOtest_metrics (dropshot): request completed latency_us = 1925 local_addr = [::1]:35099 method = GET remote_addr = [::1]:59844 req_id = bba0f5dd-27bb-47e1-995a-4bb7d9992fc8 response_code = 200 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /v2p
46212024-10-17T22:42:19.724ZINFOtest_metrics (dropshot): accepted connection local_addr = [::1]:40953 remote_addr = [::1]:57542 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
46222024-10-17T22:42:19.725ZDEBGtest_metrics (BackgroundTasks): client response SledAgent = 039be560-54cc-49e3-88df-1a29dadbf913 background_task = v2p_manager result = Ok(Response { url: "http://[::1]:35099/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "bba0f5dd-27bb-47e1-995a-4bb7d9992fc8", "content-length": "2", "date": "Thu, 17 Oct 2024 22:42:19 GMT"} })
46232024-10-17T22:42:19.725ZINFOtest_metrics (BackgroundTasks): found opte v2p mappings background_task = v2p_manager interfaces = {} sled = sim-039be560
46242024-10-17T22:42:19.725ZINFOtest_metrics (BackgroundTasks): v2p mappings to delete background_task = v2p_manager mappings = [] sled = sim-039be560
46252024-10-17T22:42:19.725ZINFOtest_metrics (BackgroundTasks): v2p mappings to add background_task = v2p_manager mappings = [] sled = sim-039be560
46262024-10-17T22:42:19.725ZDEBGtest_metrics (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = v2p_manager body = None method = GET uri = http://[::1]:40953/v2p
46272024-10-17T22:42:19.726ZTRACtest_metrics (dropshot): incoming request local_addr = [::1]:40953 method = GET remote_addr = [::1]:57542 req_id = da6abca8-7b8f-44e9-ba30-cfb429d2db05 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc-routes
46282024-10-17T22:42:19.726ZINFOtest_metrics (dropshot): accepted connection local_addr = [::1]:40953 remote_addr = [::1]:57544 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
46292024-10-17T22:42:19.729ZINFOtest_metrics (dropshot): request completed latency_us = 592 local_addr = [::1]:40953 method = GET remote_addr = [::1]:57542 req_id = da6abca8-7b8f-44e9-ba30-cfb429d2db05 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc-routes
46302024-10-17T22:42:19.729ZTRACtest_metrics (dropshot): incoming request local_addr = [::1]:40953 method = GET remote_addr = [::1]:57544 req_id = 64ca6eb0-61c1-4cf6-aa55-afef7aedca77 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /v2p
46312024-10-17T22:42:19.729ZDEBGtest_metrics (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager result = Ok(Response { url: "http://[::1]:40953/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "da6abca8-7b8f-44e9-ba30-cfb429d2db05", "content-length": "2", "date": "Thu, 17 Oct 2024 22:42:19 GMT"} })
46322024-10-17T22:42:19.729ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
46332024-10-17T22:42:19.729ZDEBGtest_metrics (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
46342024-10-17T22:42:19.730ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
46352024-10-17T22:42:19.730ZINFOtest_metrics (dropshot): request completed latency_us = 1175 local_addr = [::1]:40953 method = GET remote_addr = [::1]:57544 req_id = 64ca6eb0-61c1-4cf6-aa55-afef7aedca77 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /v2p
46362024-10-17T22:42:19.741ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database
46372024-10-17T22:42:19.741ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
46382024-10-17T22:42:19.742ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
46392024-10-17T22:42:19.744ZDEBGtest_metrics (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = v2p_manager result = Ok(Response { url: "http://[::1]:40953/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "64ca6eb0-61c1-4cf6-aa55-afef7aedca77", "content-length": "2", "date": "Thu, 17 Oct 2024 22:42:19 GMT"} })
46402024-10-17T22:42:19.744ZINFOtest_metrics (BackgroundTasks): found opte v2p mappings background_task = v2p_manager interfaces = {} sled = sim-b6d65341
46412024-10-17T22:42:19.744ZINFOtest_metrics (BackgroundTasks): v2p mappings to delete background_task = v2p_manager mappings = [] sled = sim-b6d65341
46422024-10-17T22:42:19.744ZINFOtest_metrics (BackgroundTasks): v2p mappings to add background_task = v2p_manager mappings = [] sled = sim-b6d65341
46432024-10-17T22:42:19.744ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 454.511395ms iteration = 2 status = {}
46442024-10-17T22:42:19.747ZINFOtest_metrics (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager sled = sim-b6d65341
46452024-10-17T22:42:19.747ZDEBGtest_metrics (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[::1]:40953/eip-gateways
46462024-10-17T22:42:19.747ZTRACtest_metrics (dropshot): incoming request local_addr = [::1]:40953 method = PUT remote_addr = [::1]:57542 req_id = 4a0277ac-aa7e-45f8-b944-9ab692195617 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /eip-gateways
46472024-10-17T22:42:19.747ZINFOtest_metrics (dropshot): request completed latency_us = 243 local_addr = [::1]:40953 method = PUT remote_addr = [::1]:57542 req_id = 4a0277ac-aa7e-45f8-b944-9ab692195617 response_code = 204 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /eip-gateways
46482024-10-17T22:42:19.747ZDEBGtest_metrics (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager result = Ok(Response { url: "http://[::1]:40953/eip-gateways", status: 204, headers: {"x-request-id": "4a0277ac-aa7e-45f8-b944-9ab692195617", "date": "Thu, 17 Oct 2024 22:42:19 GMT"} })
46492024-10-17T22:42:19.747ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 403.681236ms iteration = 2 status = {}
46502024-10-17T22:42:19.748ZTRACtest_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 039be560-54cc-49e3-88df-1a29dadbf913, lookup_type: ById(039be560-54cc-49e3-88df-1a29dadbf913) }
46512024-10-17T22:42:19.748ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_zone_nat_tracker resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
46522024-10-17T22:42:19.749ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database
46532024-10-17T22:42:19.749ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
46542024-10-17T22:42:19.749ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
46552024-10-17T22:42:19.752ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46562024-10-17T22:42:19.758ZDEBGtest_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 039be560-54cc-49e3-88df-1a29dadbf913, lookup_type: ById(039be560-54cc-49e3-88df-1a29dadbf913) } result = Ok(())
46572024-10-17T22:42:19.761ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database
46582024-10-17T22:42:19.763ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
46592024-10-17T22:42:19.763ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
46602024-10-17T22:42:19.764ZTRACtest_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b6d65341-167c-41df-9b5c-41cded99c229, lookup_type: ById(b6d65341-167c-41df-9b5c-41cded99c229) }
46612024-10-17T22:42:19.764ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_zone_nat_tracker resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
46622024-10-17T22:42:19.764ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database
46632024-10-17T22:42:19.765ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
46642024-10-17T22:42:19.765ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
46652024-10-17T22:42:19.766ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46662024-10-17T22:42:19.773ZDEBGtest_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b6d65341-167c-41df-9b5c-41cded99c229, lookup_type: ById(b6d65341-167c-41df-9b5c-41cded99c229) } result = Ok(())
46672024-10-17T22:42:19.773ZERROtest_metrics (BackgroundTasks): ipv6 addresses for service zone nat not implemented background_task = service_zone_nat_tracker
46682024-10-17T22:42:19.773ZERROtest_metrics (BackgroundTasks): generated config for fewer than the minimum allowed number of dns zones background_task = service_zone_nat_tracker
46692024-10-17T22:42:19.773ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 485.267892ms iteration = 2 status = {"error":"generated config for fewer than the minimum allowed number of dns zones"}
46702024-10-17T22:42:20.190ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
46712024-10-17T22:42:20.190ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
46722024-10-17T22:42:20.243ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
46732024-10-17T22:42:20.243ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
46742024-10-17T22:42:20.245ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
46752024-10-17T22:42:20.250ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
46762024-10-17T22:42:20.251ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:20.188205473","datum":22}\n
46772024-10-17T22:42:20.256ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
46782024-10-17T22:42:20.258ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
46792024-10-17T22:42:21.189ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
46802024-10-17T22:42:21.189ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
46812024-10-17T22:42:21.243ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
46822024-10-17T22:42:21.243ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
46832024-10-17T22:42:21.244ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
46842024-10-17T22:42:21.255ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
46852024-10-17T22:42:21.255ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:21.187382024","datum":23}\n
46862024-10-17T22:42:21.261ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
46872024-10-17T22:42:21.262ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
46882024-10-17T22:42:21.357ZDEBGtest_metrics (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[::1]:12225/local/switch-id
46892024-10-17T22:42:21.357ZDEBGtest_metrics (MgsClient): client response background_task = nat_v4_garbage_collector result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
46902024-10-17T22:42:21.357ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = nat_v4_garbage_collector reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
46912024-10-17T22:42:21.422ZDEBGtest_metrics (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[::1]:12225/local/switch-id
46922024-10-17T22:42:21.422ZDEBGtest_metrics (MgsClient): client response background_task = bfd_manager result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
46932024-10-17T22:42:21.423ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = bfd_manager reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
46942024-10-17T22:42:21.634ZDEBGtest_metrics (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc uri = http://[::1]:12225/local/switch-id
46952024-10-17T22:42:21.635ZDEBGtest_metrics (MgsClient): client response background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
46962024-10-17T22:42:21.635ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
46972024-10-17T22:42:22.186ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
46982024-10-17T22:42:22.189ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
46992024-10-17T22:42:22.243ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
47002024-10-17T22:42:22.243ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
47012024-10-17T22:42:22.244ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
47022024-10-17T22:42:22.251ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
47032024-10-17T22:42:22.251ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:22.186908494","datum":24}\n
47042024-10-17T22:42:22.263ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
47052024-10-17T22:42:22.263ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
47062024-10-17T22:42:23.189ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
47072024-10-17T22:42:23.190ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
47082024-10-17T22:42:23.244ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
47092024-10-17T22:42:23.244ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
47102024-10-17T22:42:23.245ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
47112024-10-17T22:42:23.252ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
47122024-10-17T22:42:23.254ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:23.187749214","datum":25}\n
47132024-10-17T22:42:23.261ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
47142024-10-17T22:42:23.265ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
47152024-10-17T22:42:23.358ZDEBGtest_metrics (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[::1]:12225/local/switch-id
47162024-10-17T22:42:23.358ZDEBGtest_metrics (MgsClient): client response background_task = nat_v4_garbage_collector result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
47172024-10-17T22:42:23.359ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = nat_v4_garbage_collector reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
47182024-10-17T22:42:23.424ZDEBGtest_metrics (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[::1]:12225/local/switch-id
47192024-10-17T22:42:23.424ZDEBGtest_metrics (MgsClient): client response background_task = bfd_manager result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
47202024-10-17T22:42:23.426ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = bfd_manager reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
47212024-10-17T22:42:23.635ZDEBGtest_metrics (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc uri = http://[::1]:12225/local/switch-id
47222024-10-17T22:42:23.637ZDEBGtest_metrics (MgsClient): client response background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
47232024-10-17T22:42:23.637ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
47242024-10-17T22:42:24.186ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
47252024-10-17T22:42:24.190ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
47262024-10-17T22:42:24.243ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
47272024-10-17T22:42:24.243ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
47282024-10-17T22:42:24.245ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
47292024-10-17T22:42:24.262ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
47302024-10-17T22:42:24.262ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:24.187588891","datum":26}\n
47312024-10-17T22:42:24.269ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
47322024-10-17T22:42:24.269ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
47332024-10-17T22:42:25.190ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
47342024-10-17T22:42:25.190ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
47352024-10-17T22:42:25.250ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
47362024-10-17T22:42:25.250ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
47372024-10-17T22:42:25.251ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
47382024-10-17T22:42:25.260ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
47392024-10-17T22:42:25.260ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:25.187703665","datum":27}\n
47402024-10-17T22:42:25.265ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
47412024-10-17T22:42:25.265ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
47422024-10-17T22:42:25.359ZDEBGtest_metrics (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[::1]:12225/local/switch-id
47432024-10-17T22:42:25.360ZDEBGtest_metrics (MgsClient): client response background_task = nat_v4_garbage_collector result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
47442024-10-17T22:42:25.360ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = nat_v4_garbage_collector reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
47452024-10-17T22:42:25.426ZDEBGtest_metrics (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[::1]:12225/local/switch-id
47462024-10-17T22:42:25.426ZDEBGtest_metrics (MgsClient): client response background_task = bfd_manager result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
47472024-10-17T22:42:25.426ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = bfd_manager reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
47482024-10-17T22:42:25.637ZDEBGtest_metrics (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc uri = http://[::1]:12225/local/switch-id
47492024-10-17T22:42:25.640ZDEBGtest_metrics (MgsClient): client response background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
47502024-10-17T22:42:25.640ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
47512024-10-17T22:42:26.187ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
47522024-10-17T22:42:26.189ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
47532024-10-17T22:42:26.243ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
47542024-10-17T22:42:26.244ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
47552024-10-17T22:42:26.245ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
47562024-10-17T22:42:26.249ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
47572024-10-17T22:42:26.249ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:26.187459279","datum":28}\n
47582024-10-17T22:42:26.253ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
47592024-10-17T22:42:26.253ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
47602024-10-17T22:42:26.651ZINFOtest_metrics (dropshot): accepted connection kind = pantry local_addr = [::1]:38587 remote_addr = [::1]:42120 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
47612024-10-17T22:42:26.651ZTRACtest_metrics (dropshot): incoming request kind = pantry local_addr = [::1]:38587 method = GET remote_addr = [::1]:42120 req_id = a2bc2038-886f-4ab2-9b06-584c769fb1e8 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
47622024-10-17T22:42:26.652ZINFOtest_metrics (dropshot): request completed kind = pantry latency_us = 210 local_addr = [::1]:38587 method = GET remote_addr = [::1]:42120 req_id = a2bc2038-886f-4ab2-9b06-584c769fb1e8 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
47632024-10-17T22:42:26.692ZINFOtest_metrics (dropshot): accepted connection kind = pantry local_addr = [::1]:38587 remote_addr = [::1]:42122 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
47642024-10-17T22:42:26.692ZTRACtest_metrics (dropshot): incoming request kind = pantry local_addr = [::1]:38587 method = GET remote_addr = [::1]:42122 req_id = 3e9d09f8-2092-4135-9ef6-1d85966192dc sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
47652024-10-17T22:42:26.692ZINFOtest_metrics (dropshot): request completed kind = pantry latency_us = 180 local_addr = [::1]:38587 method = GET remote_addr = [::1]:42122 req_id = 3e9d09f8-2092-4135-9ef6-1d85966192dc response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
47662024-10-17T22:42:26.740ZINFOtest_metrics (dropshot): accepted connection kind = pantry local_addr = [::1]:38587 remote_addr = [::1]:42124 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
47672024-10-17T22:42:26.740ZTRACtest_metrics (dropshot): incoming request kind = pantry local_addr = [::1]:38587 method = GET remote_addr = [::1]:42124 req_id = e1c7f945-5ddb-4952-9b11-1d1e7f76e024 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
47682024-10-17T22:42:26.741ZINFOtest_metrics (dropshot): request completed kind = pantry latency_us = 250 local_addr = [::1]:38587 method = GET remote_addr = [::1]:42124 req_id = e1c7f945-5ddb-4952-9b11-1d1e7f76e024 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
47692024-10-17T22:42:26.820ZINFOtest_metrics (dropshot): accepted connection kind = pantry local_addr = [::1]:38587 remote_addr = [::1]:42126 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
47702024-10-17T22:42:26.820ZTRACtest_metrics (dropshot): incoming request kind = pantry local_addr = [::1]:38587 method = GET remote_addr = [::1]:42126 req_id = ed44e76d-6144-4270-a516-e7eee59824dc sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
47712024-10-17T22:42:26.820ZINFOtest_metrics (dropshot): request completed kind = pantry latency_us = 225 local_addr = [::1]:38587 method = GET remote_addr = [::1]:42126 req_id = ed44e76d-6144-4270-a516-e7eee59824dc response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
47722024-10-17T22:42:26.822ZINFOtest_metrics (dropshot): accepted connection kind = pantry local_addr = [::1]:38587 remote_addr = [::1]:42128 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
47732024-10-17T22:42:26.823ZTRACtest_metrics (dropshot): incoming request kind = pantry local_addr = [::1]:38587 method = GET remote_addr = [::1]:42128 req_id = 5b993010-af47-4043-9823-008791799dd2 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
47742024-10-17T22:42:26.823ZINFOtest_metrics (dropshot): request completed kind = pantry latency_us = 166 local_addr = [::1]:38587 method = GET remote_addr = [::1]:42128 req_id = 5b993010-af47-4043-9823-008791799dd2 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
47752024-10-17T22:42:26.853ZINFOtest_metrics (dropshot): accepted connection kind = pantry local_addr = [::1]:38587 remote_addr = [::1]:42130 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
47762024-10-17T22:42:26.854ZTRACtest_metrics (dropshot): incoming request kind = pantry local_addr = [::1]:38587 method = GET remote_addr = [::1]:42130 req_id = 63283180-4cff-4a67-8c18-5ab4f0c6f6f1 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
47772024-10-17T22:42:26.854ZINFOtest_metrics (dropshot): request completed kind = pantry latency_us = 242 local_addr = [::1]:38587 method = GET remote_addr = [::1]:42130 req_id = 63283180-4cff-4a67-8c18-5ab4f0c6f6f1 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
47782024-10-17T22:42:26.878ZINFOtest_metrics (dropshot): accepted connection kind = pantry local_addr = [::1]:38587 remote_addr = [::1]:42132 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
47792024-10-17T22:42:26.879ZTRACtest_metrics (dropshot): incoming request kind = pantry local_addr = [::1]:38587 method = GET remote_addr = [::1]:42132 req_id = b66ef271-5bd4-4afe-8e6f-d30c78d7a5b5 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
47802024-10-17T22:42:26.879ZINFOtest_metrics (dropshot): request completed kind = pantry latency_us = 208 local_addr = [::1]:38587 method = GET remote_addr = [::1]:42132 req_id = b66ef271-5bd4-4afe-8e6f-d30c78d7a5b5 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
47812024-10-17T22:42:26.922ZINFOtest_metrics (dropshot): accepted connection kind = pantry local_addr = [::1]:38587 remote_addr = [::1]:42134 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
47822024-10-17T22:42:26.923ZTRACtest_metrics (dropshot): incoming request kind = pantry local_addr = [::1]:38587 method = GET remote_addr = [::1]:42134 req_id = 51474c9e-84cb-4311-bc10-4ac3cccd79f8 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
47832024-10-17T22:42:26.923ZINFOtest_metrics (dropshot): request completed kind = pantry latency_us = 226 local_addr = [::1]:38587 method = GET remote_addr = [::1]:42134 req_id = 51474c9e-84cb-4311-bc10-4ac3cccd79f8 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
47842024-10-17T22:42:27.186ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
47852024-10-17T22:42:27.189ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
47862024-10-17T22:42:27.243ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
47872024-10-17T22:42:27.243ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
47882024-10-17T22:42:27.244ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
47892024-10-17T22:42:27.250ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
47902024-10-17T22:42:27.252ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:27.186957048","datum":29}\n
47912024-10-17T22:42:27.259ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
47922024-10-17T22:42:27.259ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
47932024-10-17T22:42:27.361ZDEBGtest_metrics (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[::1]:12225/local/switch-id
47942024-10-17T22:42:27.361ZDEBGtest_metrics (MgsClient): client response background_task = nat_v4_garbage_collector result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
47952024-10-17T22:42:27.361ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = nat_v4_garbage_collector reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
47962024-10-17T22:42:27.427ZDEBGtest_metrics (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[::1]:12225/local/switch-id
47972024-10-17T22:42:27.427ZDEBGtest_metrics (MgsClient): client response background_task = bfd_manager result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
47982024-10-17T22:42:27.427ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = bfd_manager reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
47992024-10-17T22:42:27.638ZDEBGtest_metrics (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc uri = http://[::1]:12225/local/switch-id
48002024-10-17T22:42:27.639ZDEBGtest_metrics (MgsClient): client response background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
48012024-10-17T22:42:27.640ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
48022024-10-17T22:42:28.187ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
48032024-10-17T22:42:28.189ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
48042024-10-17T22:42:28.244ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
48052024-10-17T22:42:28.244ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
48062024-10-17T22:42:28.245ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
48072024-10-17T22:42:28.253ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
48082024-10-17T22:42:28.253ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:28.187769508","datum":30}\n
48092024-10-17T22:42:28.259ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
48102024-10-17T22:42:28.260ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
48112024-10-17T22:42:29.192ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
48122024-10-17T22:42:29.196ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
48132024-10-17T22:42:29.244ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
48142024-10-17T22:42:29.244ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
48152024-10-17T22:42:29.246ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
48162024-10-17T22:42:29.256ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
48172024-10-17T22:42:29.256ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:29.189100691","datum":31}\n
48182024-10-17T22:42:29.261ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
48192024-10-17T22:42:29.262ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
48202024-10-17T22:42:29.363ZDEBGtest_metrics (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[::1]:12225/local/switch-id
48212024-10-17T22:42:29.363ZDEBGtest_metrics (MgsClient): client response background_task = nat_v4_garbage_collector result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
48222024-10-17T22:42:29.364ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = nat_v4_garbage_collector reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
48232024-10-17T22:42:29.429ZDEBGtest_metrics (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[::1]:12225/local/switch-id
48242024-10-17T22:42:29.429ZDEBGtest_metrics (MgsClient): client response background_task = bfd_manager result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
48252024-10-17T22:42:29.429ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = bfd_manager reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
48262024-10-17T22:42:29.641ZDEBGtest_metrics (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc uri = http://[::1]:12225/local/switch-id
48272024-10-17T22:42:29.641ZDEBGtest_metrics (MgsClient): client response background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
48282024-10-17T22:42:29.641ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
48292024-10-17T22:42:30.186ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
48302024-10-17T22:42:30.189ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
48312024-10-17T22:42:30.243ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
48322024-10-17T22:42:30.243ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
48332024-10-17T22:42:30.244ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
48342024-10-17T22:42:30.251ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
48352024-10-17T22:42:30.251ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:30.187109272","datum":32}\n
48362024-10-17T22:42:30.259ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
48372024-10-17T22:42:30.259ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
48382024-10-17T22:42:31.187ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
48392024-10-17T22:42:31.189ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
48402024-10-17T22:42:31.244ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
48412024-10-17T22:42:31.244ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
48422024-10-17T22:42:31.245ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
48432024-10-17T22:42:31.250ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
48442024-10-17T22:42:31.250ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:31.187497219","datum":33}\n
48452024-10-17T22:42:31.254ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
48462024-10-17T22:42:31.254ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
48472024-10-17T22:42:31.365ZDEBGtest_metrics (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[::1]:12225/local/switch-id
48482024-10-17T22:42:31.365ZDEBGtest_metrics (MgsClient): client response background_task = nat_v4_garbage_collector result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
48492024-10-17T22:42:31.365ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = nat_v4_garbage_collector reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
48502024-10-17T22:42:31.431ZDEBGtest_metrics (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[::1]:12225/local/switch-id
48512024-10-17T22:42:31.431ZDEBGtest_metrics (MgsClient): client response background_task = bfd_manager result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
48522024-10-17T22:42:31.431ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = bfd_manager reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
48532024-10-17T22:42:31.643ZDEBGtest_metrics (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc uri = http://[::1]:12225/local/switch-id
48542024-10-17T22:42:31.643ZDEBGtest_metrics (MgsClient): client response background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
48552024-10-17T22:42:31.643ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
48562024-10-17T22:42:32.186ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
48572024-10-17T22:42:32.189ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
48582024-10-17T22:42:32.244ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
48592024-10-17T22:42:32.244ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
48602024-10-17T22:42:32.245ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
48612024-10-17T22:42:32.251ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
48622024-10-17T22:42:32.252ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:32.187552519","datum":34}\n
48632024-10-17T22:42:32.256ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
48642024-10-17T22:42:32.256ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
48652024-10-17T22:42:33.186ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
48662024-10-17T22:42:33.189ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
48672024-10-17T22:42:33.249ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
48682024-10-17T22:42:33.249ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
48692024-10-17T22:42:33.249ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
48702024-10-17T22:42:33.255ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
48712024-10-17T22:42:33.255ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:33.186833710","datum":35}\n
48722024-10-17T22:42:33.260ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
48732024-10-17T22:42:33.260ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
48742024-10-17T22:42:33.367ZDEBGtest_metrics (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[::1]:12225/local/switch-id
48752024-10-17T22:42:33.370ZDEBGtest_metrics (MgsClient): client response background_task = nat_v4_garbage_collector result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
48762024-10-17T22:42:33.370ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = nat_v4_garbage_collector reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
48772024-10-17T22:42:33.433ZDEBGtest_metrics (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[::1]:12225/local/switch-id
48782024-10-17T22:42:33.433ZDEBGtest_metrics (MgsClient): client response background_task = bfd_manager result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
48792024-10-17T22:42:33.433ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = bfd_manager reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
48802024-10-17T22:42:33.644ZDEBGtest_metrics (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc uri = http://[::1]:12225/local/switch-id
48812024-10-17T22:42:33.645ZDEBGtest_metrics (MgsClient): client response background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
48822024-10-17T22:42:33.645ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
48832024-10-17T22:42:34.187ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
48842024-10-17T22:42:34.190ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
48852024-10-17T22:42:34.243ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
48862024-10-17T22:42:34.243ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
48872024-10-17T22:42:34.244ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
48882024-10-17T22:42:34.257ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
48892024-10-17T22:42:34.258ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:34.187700838","datum":36}\n
48902024-10-17T22:42:34.268ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
48912024-10-17T22:42:34.268ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
48922024-10-17T22:42:35.186ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
48932024-10-17T22:42:35.189ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
48942024-10-17T22:42:35.243ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
48952024-10-17T22:42:35.243ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
48962024-10-17T22:42:35.244ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
48972024-10-17T22:42:35.248ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
48982024-10-17T22:42:35.249ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:35.187009500","datum":37}\n
48992024-10-17T22:42:35.252ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
49002024-10-17T22:42:35.253ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
49012024-10-17T22:42:35.368ZDEBGtest_metrics (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[::1]:12225/local/switch-id
49022024-10-17T22:42:35.369ZDEBGtest_metrics (MgsClient): client response background_task = nat_v4_garbage_collector result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
49032024-10-17T22:42:35.369ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = nat_v4_garbage_collector reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
49042024-10-17T22:42:35.435ZDEBGtest_metrics (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[::1]:12225/local/switch-id
49052024-10-17T22:42:35.435ZDEBGtest_metrics (MgsClient): client response background_task = bfd_manager result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
49062024-10-17T22:42:35.435ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = bfd_manager reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
49072024-10-17T22:42:35.647ZDEBGtest_metrics (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc uri = http://[::1]:12225/local/switch-id
49082024-10-17T22:42:35.647ZDEBGtest_metrics (MgsClient): client response background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
49092024-10-17T22:42:35.647ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
49102024-10-17T22:42:36.187ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
49112024-10-17T22:42:36.198ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
49122024-10-17T22:42:36.243ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
49132024-10-17T22:42:36.244ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
49142024-10-17T22:42:36.244ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
49152024-10-17T22:42:36.249ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
49162024-10-17T22:42:36.250ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:36.187453099","datum":38}\n
49172024-10-17T22:42:36.253ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
49182024-10-17T22:42:36.253ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
49192024-10-17T22:42:37.187ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
49202024-10-17T22:42:37.189ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
49212024-10-17T22:42:37.246ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
49222024-10-17T22:42:37.248ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
49232024-10-17T22:42:37.248ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
49242024-10-17T22:42:37.257ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
49252024-10-17T22:42:37.257ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:37.187634632","datum":39}\n
49262024-10-17T22:42:37.262ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
49272024-10-17T22:42:37.262ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
49282024-10-17T22:42:37.371ZDEBGtest_metrics (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[::1]:12225/local/switch-id
49292024-10-17T22:42:37.371ZDEBGtest_metrics (MgsClient): client response background_task = nat_v4_garbage_collector result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
49302024-10-17T22:42:37.371ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = nat_v4_garbage_collector reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
49312024-10-17T22:42:37.436ZDEBGtest_metrics (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[::1]:12225/local/switch-id
49322024-10-17T22:42:37.437ZDEBGtest_metrics (MgsClient): client response background_task = bfd_manager result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
49332024-10-17T22:42:37.438ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = bfd_manager reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
49342024-10-17T22:42:37.648ZDEBGtest_metrics (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc uri = http://[::1]:12225/local/switch-id
49352024-10-17T22:42:37.649ZDEBGtest_metrics (MgsClient): client response background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
49362024-10-17T22:42:37.649ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
49372024-10-17T22:42:38.187ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
49382024-10-17T22:42:38.189ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
49392024-10-17T22:42:38.243ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
49402024-10-17T22:42:38.243ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
49412024-10-17T22:42:38.246ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
49422024-10-17T22:42:38.253ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
49432024-10-17T22:42:38.253ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:38.187363283","datum":40}\n
49442024-10-17T22:42:38.257ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
49452024-10-17T22:42:38.258ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
49462024-10-17T22:42:39.186ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
49472024-10-17T22:42:39.189ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
49482024-10-17T22:42:39.243ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
49492024-10-17T22:42:39.244ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
49502024-10-17T22:42:39.245ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
49512024-10-17T22:42:39.250ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
49522024-10-17T22:42:39.250ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:39.187246568","datum":41}\n
49532024-10-17T22:42:39.256ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
49542024-10-17T22:42:39.258ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
49552024-10-17T22:42:39.373ZDEBGtest_metrics (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[::1]:12225/local/switch-id
49562024-10-17T22:42:39.373ZDEBGtest_metrics (MgsClient): client response background_task = nat_v4_garbage_collector result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
49572024-10-17T22:42:39.373ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = nat_v4_garbage_collector reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
49582024-10-17T22:42:39.438ZDEBGtest_metrics (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[::1]:12225/local/switch-id
49592024-10-17T22:42:39.438ZDEBGtest_metrics (MgsClient): client response background_task = bfd_manager result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
49602024-10-17T22:42:39.438ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = bfd_manager reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
49612024-10-17T22:42:39.651ZDEBGtest_metrics (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc uri = http://[::1]:12225/local/switch-id
49622024-10-17T22:42:39.651ZDEBGtest_metrics (MgsClient): client response background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
49632024-10-17T22:42:39.651ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
49642024-10-17T22:42:40.187ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
49652024-10-17T22:42:40.189ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
49662024-10-17T22:42:40.243ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
49672024-10-17T22:42:40.244ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
49682024-10-17T22:42:40.244ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
49692024-10-17T22:42:40.248ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
49702024-10-17T22:42:40.250ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:40.187458692","datum":42}\n
49712024-10-17T22:42:40.253ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
49722024-10-17T22:42:40.254ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
49732024-10-17T22:42:41.191ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
49742024-10-17T22:42:41.191ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
49752024-10-17T22:42:41.244ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
49762024-10-17T22:42:41.244ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
49772024-10-17T22:42:41.245ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
49782024-10-17T22:42:41.250ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
49792024-10-17T22:42:41.251ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:41.187666134","datum":43}\n
49802024-10-17T22:42:41.256ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
49812024-10-17T22:42:41.257ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
49822024-10-17T22:42:41.374ZDEBGtest_metrics (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[::1]:12225/local/switch-id
49832024-10-17T22:42:41.375ZDEBGtest_metrics (MgsClient): client response background_task = nat_v4_garbage_collector result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
49842024-10-17T22:42:41.375ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = nat_v4_garbage_collector reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
49852024-10-17T22:42:41.439ZDEBGtest_metrics (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[::1]:12225/local/switch-id
49862024-10-17T22:42:41.440ZDEBGtest_metrics (MgsClient): client response background_task = bfd_manager result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
49872024-10-17T22:42:41.440ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = bfd_manager reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
49882024-10-17T22:42:41.652ZDEBGtest_metrics (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc uri = http://[::1]:12225/local/switch-id
49892024-10-17T22:42:41.653ZDEBGtest_metrics (MgsClient): client response background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
49902024-10-17T22:42:41.653ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
49912024-10-17T22:42:42.187ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
49922024-10-17T22:42:42.190ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
49932024-10-17T22:42:42.245ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
49942024-10-17T22:42:42.245ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
49952024-10-17T22:42:42.245ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
49962024-10-17T22:42:42.249ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
49972024-10-17T22:42:42.249ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:42.188173145","datum":44}\n
49982024-10-17T22:42:42.259ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
49992024-10-17T22:42:42.259ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
50002024-10-17T22:42:43.189ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
50012024-10-17T22:42:43.190ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
50022024-10-17T22:42:43.244ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
50032024-10-17T22:42:43.245ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
50042024-10-17T22:42:43.245ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
50052024-10-17T22:42:43.259ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
50062024-10-17T22:42:43.259ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:43.187472466","datum":45}\n
50072024-10-17T22:42:43.287ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
50082024-10-17T22:42:43.289ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
50092024-10-17T22:42:43.376ZDEBGtest_metrics (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[::1]:12225/local/switch-id
50102024-10-17T22:42:43.376ZDEBGtest_metrics (MgsClient): client response background_task = nat_v4_garbage_collector result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
50112024-10-17T22:42:43.377ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = nat_v4_garbage_collector reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
50122024-10-17T22:42:43.441ZDEBGtest_metrics (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[::1]:12225/local/switch-id
50132024-10-17T22:42:43.442ZDEBGtest_metrics (MgsClient): client response background_task = bfd_manager result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
50142024-10-17T22:42:43.442ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = bfd_manager reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
50152024-10-17T22:42:43.654ZDEBGtest_metrics (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc uri = http://[::1]:12225/local/switch-id
50162024-10-17T22:42:43.654ZDEBGtest_metrics (MgsClient): client response background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
50172024-10-17T22:42:43.655ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
50182024-10-17T22:42:44.186ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
50192024-10-17T22:42:44.189ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
50202024-10-17T22:42:44.243ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
50212024-10-17T22:42:44.243ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
50222024-10-17T22:42:44.244ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
50232024-10-17T22:42:44.250ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
50242024-10-17T22:42:44.250ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:44.187102565","datum":46}\n
50252024-10-17T22:42:44.259ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
50262024-10-17T22:42:44.261ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
50272024-10-17T22:42:45.186ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
50282024-10-17T22:42:45.189ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
50292024-10-17T22:42:45.243ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
50302024-10-17T22:42:45.244ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
50312024-10-17T22:42:45.244ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
50322024-10-17T22:42:45.249ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
50332024-10-17T22:42:45.250ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:45.186935199","datum":47}\n
50342024-10-17T22:42:45.254ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
50352024-10-17T22:42:45.254ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
50362024-10-17T22:42:45.377ZDEBGtest_metrics (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[::1]:12225/local/switch-id
50372024-10-17T22:42:45.378ZDEBGtest_metrics (MgsClient): client response background_task = nat_v4_garbage_collector result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
50382024-10-17T22:42:45.378ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = nat_v4_garbage_collector reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
50392024-10-17T22:42:45.442ZDEBGtest_metrics (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[::1]:12225/local/switch-id
50402024-10-17T22:42:45.443ZDEBGtest_metrics (MgsClient): client response background_task = bfd_manager result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
50412024-10-17T22:42:45.443ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = bfd_manager reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
50422024-10-17T22:42:45.655ZDEBGtest_metrics (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc uri = http://[::1]:12225/local/switch-id
50432024-10-17T22:42:45.656ZDEBGtest_metrics (MgsClient): client response background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
50442024-10-17T22:42:45.656ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
50452024-10-17T22:42:46.189ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
50462024-10-17T22:42:46.190ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
50472024-10-17T22:42:46.245ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
50482024-10-17T22:42:46.248ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
50492024-10-17T22:42:46.248ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
50502024-10-17T22:42:46.253ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
50512024-10-17T22:42:46.255ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:46.187741620","datum":48}\n
50522024-10-17T22:42:46.260ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
50532024-10-17T22:42:46.260ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
50542024-10-17T22:42:47.186ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
50552024-10-17T22:42:47.188ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
50562024-10-17T22:42:47.244ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
50572024-10-17T22:42:47.244ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
50582024-10-17T22:42:47.245ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
50592024-10-17T22:42:47.252ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
50602024-10-17T22:42:47.252ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:47.186861417","datum":49}\n
50612024-10-17T22:42:47.259ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
50622024-10-17T22:42:47.259ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
50632024-10-17T22:42:47.379ZDEBGtest_metrics (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[::1]:12225/local/switch-id
50642024-10-17T22:42:47.379ZDEBGtest_metrics (MgsClient): client response background_task = nat_v4_garbage_collector result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
50652024-10-17T22:42:47.379ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = nat_v4_garbage_collector reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
50662024-10-17T22:42:47.443ZDEBGtest_metrics (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[::1]:12225/local/switch-id
50672024-10-17T22:42:47.444ZDEBGtest_metrics (MgsClient): client response background_task = bfd_manager result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
50682024-10-17T22:42:47.444ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = bfd_manager reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
50692024-10-17T22:42:47.656ZDEBGtest_metrics (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc uri = http://[::1]:12225/local/switch-id
50702024-10-17T22:42:47.657ZDEBGtest_metrics (MgsClient): client response background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
50712024-10-17T22:42:47.657ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
50722024-10-17T22:42:48.187ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
50732024-10-17T22:42:48.190ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
50742024-10-17T22:42:48.243ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
50752024-10-17T22:42:48.243ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
50762024-10-17T22:42:48.244ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
50772024-10-17T22:42:48.249ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
50782024-10-17T22:42:48.249ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:48.187854960","datum":50}\n
50792024-10-17T22:42:48.253ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
50802024-10-17T22:42:48.254ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
50812024-10-17T22:42:49.187ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
50822024-10-17T22:42:49.191ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
50832024-10-17T22:42:49.245ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
50842024-10-17T22:42:49.245ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
50852024-10-17T22:42:49.245ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
50862024-10-17T22:42:49.252ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
50872024-10-17T22:42:49.254ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:49.188033343","datum":51}\n
50882024-10-17T22:42:49.259ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
50892024-10-17T22:42:49.259ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
50902024-10-17T22:42:49.268ZDEBGtest_metrics (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 3 reason = Timeout
50912024-10-17T22:42:49.272ZDEBGtest_metrics (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
50922024-10-17T22:42:49.272ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
50932024-10-17T22:42:49.272ZDEBGtest_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
50942024-10-17T22:42:49.272ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
50952024-10-17T22:42:49.272ZDEBGtest_metrics (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 3 reason = Timeout
50962024-10-17T22:42:49.273ZTRACtest_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal
50972024-10-17T22:42:49.273ZTRACtest_metrics (dns): buffer buffer = Length: 75 (0x4b) bytes\n0000: 5c 09 01 00 00 01 00 00 00 00 00 01 0c 5f 6e 61 \\............_na\n0010: 6d 65 73 65 72 76 69 63 65 04 5f 74 63 70 0d 63 meservice._tcp.c\n0020: 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 ontrol-plane.oxi\n0030: 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 de.internal..!..\n0040: 00 00 29 04 d0 00 00 00 00 00 00 ..)........ kind = dns peer_addr = [::1]:55807 req_id = ed44379a-1b26-465a-82ab-98b6b2dd5dd0
50982024-10-17T22:42:49.273ZDEBGtest_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 23561,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_nameservice._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_nameservice._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 12,\n 95,\n 110,\n 97,\n 109,\n 101,\n 115,\n 101,\n 114,\n 118,\n 105,\n 99,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:55807 req_id = ed44379a-1b26-465a-82ab-98b6b2dd5dd0
50992024-10-17T22:42:49.273ZDEBGtest_metrics (store): query key key = _nameservice._tcp
51002024-10-17T22:42:49.273ZDEBGtest_metrics (store): query key key = 996eea1f-0c18-450c-8228-39db7dc3685a.host
51012024-10-17T22:42:49.273ZDEBGtest_metrics (dns): dns response additional_records = [Record { name_labels: Name("996eea1f-0c18-450c-8228-39db7dc3685a.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:55807 query = LowerQuery { name: LowerName(Name("_nameservice._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 41599, target: Name("996eea1f-0c18-450c-8228-39db7dc3685a.host.control-plane.oxide.internal") })) }] req_id = ed44379a-1b26-465a-82ab-98b6b2dd5dd0
51022024-10-17T22:42:49.273ZDEBGtest_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 41599, target: Name("996eea1f-0c18-450c-8228-39db7dc3685a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("996eea1f-0c18-450c-8228-39db7dc3685a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2141, tv_nsec: 40369361 } })
51032024-10-17T22:42:49.273ZTRACtest_metrics (DnsResolver): lookup_service_targets: looking up SRV target
51042024-10-17T22:42:49.273ZTRACtest_metrics (dns): buffer buffer = Length: 99 (0x63) bytes\n0000: 89 cb 01 00 00 01 00 00 00 00 00 01 24 39 39 36 ............$996\n0010: 65 65 61 31 66 2d 30 63 31 38 2d 34 35 30 63 2d eea1f-0c18-450c-\n0020: 38 32 32 38 2d 33 39 64 62 37 64 63 33 36 38 35 8228-39db7dc3685\n0030: 61 04 68 6f 73 74 0d 63 6f 6e 74 72 6f 6c 2d 70 a.host.control-p\n0040: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0050: 6e 61 6c 00 00 1c 00 01 00 00 29 04 d0 00 00 00 nal.......).....\n0060: 00 00 00 ... kind = dns peer_addr = [::1]:59810 req_id = 7b73b10e-131f-48df-946f-d80fe00c49db
51052024-10-17T22:42:49.273ZDEBGtest_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 35275,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("996eea1f-0c18-450c-8228-39db7dc3685a.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("996eea1f-0c18-450c-8228-39db7dc3685a.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 36,\n 57,\n 57,\n 54,\n 101,\n 101,\n 97,\n 49,\n 102,\n 45,\n 48,\n 99,\n 49,\n 56,\n 45,\n 52,\n 53,\n 48,\n 99,\n 45,\n 56,\n 50,\n 50,\n 56,\n 45,\n 51,\n 57,\n 100,\n 98,\n 55,\n 100,\n 99,\n 51,\n 54,\n 56,\n 53,\n 97,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:59810 req_id = 7b73b10e-131f-48df-946f-d80fe00c49db
51062024-10-17T22:42:49.273ZDEBGtest_metrics (store): query key key = 996eea1f-0c18-450c-8228-39db7dc3685a.host
51072024-10-17T22:42:49.274ZDEBGtest_metrics (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:59810 query = LowerQuery { name: LowerName(Name("996eea1f-0c18-450c-8228-39db7dc3685a.host.control-plane.oxide.internal.")), original: Query { name: Name("996eea1f-0c18-450c-8228-39db7dc3685a.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("996eea1f-0c18-450c-8228-39db7dc3685a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 7b73b10e-131f-48df-946f-d80fe00c49db
51082024-10-17T22:42:49.274ZDEBGtest_metrics (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[::1]:41599] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[::1]:41599] } dns_group = internal
51092024-10-17T22:42:49.274ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 2.639403ms iteration = 3 status = {"addresses":["[::1]:41599"]}
51102024-10-17T22:42:49.274ZDEBGtest_metrics (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 4 reason = Timeout
51112024-10-17T22:42:49.314ZDEBGtest_metrics (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[::1]:41599 generation = 1 method = PUT servers = DnsServersList { addresses: [[::1]:41599] } uri = http://[::1]:41599/config
51122024-10-17T22:42:49.314ZDEBGtest_metrics (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 3 reason = Timeout
51132024-10-17T22:42:49.315ZDEBGtest_metrics (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
51142024-10-17T22:42:49.315ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
51152024-10-17T22:42:49.315ZDEBGtest_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
51162024-10-17T22:42:49.316ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
51172024-10-17T22:42:49.316ZTRACtest_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig
51182024-10-17T22:42:49.317ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_internal dns_group = internal resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
51192024-10-17T22:42:49.317ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
51202024-10-17T22:42:49.317ZDEBGtest_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
51212024-10-17T22:42:49.317ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
51222024-10-17T22:42:49.318ZINFOtest_metrics (http): accepted connection kind = dns local_addr = [::1]:41599 remote_addr = [::1]:52946
51232024-10-17T22:42:49.318ZDEBGtest_metrics (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 4 reason = Timeout
51242024-10-17T22:42:49.373ZDEBGtest_metrics (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[::1]:36177 generation = 2 method = PUT servers = DnsServersList { addresses: [[::1]:36177] } uri = http://[::1]:36177/config
51252024-10-17T22:42:49.374ZDEBGtest_metrics (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 3 reason = Timeout
51262024-10-17T22:42:49.374ZTRACtest_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal
51272024-10-17T22:42:49.375ZDEBGtest_metrics (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 2 reason = Timeout
51282024-10-17T22:42:49.375ZINFOtest_metrics (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-10-17 22:32:49.374566145 UTC
51292024-10-17T22:42:49.375ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database
51302024-10-17T22:42:49.375ZDEBGtest_metrics (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
51312024-10-17T22:42:49.382ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
51322024-10-17T22:42:49.382ZDEBGtest_metrics (BackgroundTasks): activating background_task = external_endpoints iteration = 3 reason = Timeout
51332024-10-17T22:42:49.382ZTRACtest_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet
51342024-10-17T22:42:49.382ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = external_endpoints resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
51352024-10-17T22:42:49.382ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
51362024-10-17T22:42:49.383ZDEBGtest_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
51372024-10-17T22:42:49.383ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
51382024-10-17T22:42:49.384ZDEBGtest_metrics (BackgroundTasks): activating background_task = phantom_disks iteration = 3 reason = Timeout
51392024-10-17T22:42:49.384ZDEBGtest_metrics (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 2 reason = Timeout
51402024-10-17T22:42:49.384ZINFOtest_metrics (BackgroundTasks): Decommissioned Disk Cleaner disabled background_task = decommissioned_disk_cleaner
51412024-10-17T22:42:49.384ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 38.431\u{b5}s iteration = 2 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
51422024-10-17T22:42:49.384ZDEBGtest_metrics (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 8 reason = Timeout
51432024-10-17T22:42:49.384ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 10.75\u{b5}s iteration = 8 status = {"error":"task disabled"}
51442024-10-17T22:42:49.387ZDEBGtest_metrics (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 3 reason = Timeout
51452024-10-17T22:42:49.387ZTRACtest_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory
51462024-10-17T22:42:49.387ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_zone_nat_tracker resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
51472024-10-17T22:42:49.387ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database
51482024-10-17T22:42:49.387ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
51492024-10-17T22:42:49.387ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
51502024-10-17T22:42:49.387ZDEBGtest_metrics (BackgroundTasks): activating background_task = v2p_manager iteration = 3 reason = Timeout
51512024-10-17T22:42:49.388ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
51522024-10-17T22:42:49.388ZDEBGtest_metrics (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
51532024-10-17T22:42:49.388ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
51542024-10-17T22:42:49.388ZDEBGtest_metrics (BackgroundTasks): activating background_task = region_replacement iteration = 2 reason = Timeout
51552024-10-17T22:42:49.388ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database
51562024-10-17T22:42:49.388ZDEBGtest_metrics (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
51572024-10-17T22:42:49.388ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
51582024-10-17T22:42:49.388ZDEBGtest_metrics (BackgroundTasks): activating background_task = region_replacement_driver iteration = 2 reason = Timeout
51592024-10-17T22:42:49.388ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database
51602024-10-17T22:42:49.389ZDEBGtest_metrics (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
51612024-10-17T22:42:49.389ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
51622024-10-17T22:42:49.389ZDEBGtest_metrics (BackgroundTasks): activating background_task = instance_watcher iteration = 3 reason = Timeout
51632024-10-17T22:42:49.389ZTRACtest_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet
51642024-10-17T22:42:49.389ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = instance_watcher resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
51652024-10-17T22:42:49.389ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database
51662024-10-17T22:42:49.389ZDEBGtest_metrics (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
51672024-10-17T22:42:49.389ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
51682024-10-17T22:42:49.389ZDEBGtest_metrics (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 2 reason = Timeout
51692024-10-17T22:42:49.389ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database
51702024-10-17T22:42:49.389ZDEBGtest_metrics (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
51712024-10-17T22:42:49.389ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
51722024-10-17T22:42:49.390ZDEBGtest_metrics (BackgroundTasks): activating background_task = lookup_region_port iteration = 2 reason = Timeout
51732024-10-17T22:42:49.392ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database
51742024-10-17T22:42:49.393ZDEBGtest_metrics (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
51752024-10-17T22:42:49.393ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
51762024-10-17T22:42:49.393ZDEBGtest_metrics (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 2 reason = Timeout
51772024-10-17T22:42:49.393ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 54.851\u{b5}s iteration = 2 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
51782024-10-17T22:42:49.393ZDEBGtest_metrics (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 2 reason = Timeout
51792024-10-17T22:42:49.393ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 11.99\u{b5}s iteration = 2 status = {"errors":[],"garbage_collect_requested":[]}
51802024-10-17T22:42:49.393ZDEBGtest_metrics (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 2 reason = Timeout
51812024-10-17T22:42:49.393ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 14.24\u{b5}s iteration = 2 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
51822024-10-17T22:42:49.394ZDEBGtest_metrics (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 2 reason = Timeout
51832024-10-17T22:42:49.394ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 10.75\u{b5}s iteration = 2 status = {"errors":[],"records_set_to_done":[]}
51842024-10-17T22:42:49.394ZTRACtest_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig
51852024-10-17T22:42:49.394ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_external dns_group = external resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
51862024-10-17T22:42:49.394ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
51872024-10-17T22:42:49.395ZDEBGtest_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
51882024-10-17T22:42:49.395ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
51892024-10-17T22:42:49.395ZTRACtest_metrics (http): incoming request kind = dns local_addr = [::1]:41599 method = PUT remote_addr = [::1]:52946 req_id = 4cc3bee4-ffce-4bc8-8095-10514b4d1dda uri = /config
51902024-10-17T22:42:49.395ZINFOtest_metrics (http): accepted connection kind = dns local_addr = [::1]:36177 remote_addr = [::1]:48052
51912024-10-17T22:42:49.395ZDEBGtest_metrics (BackgroundTasks): activating background_task = instance_updater iteration = 3 reason = Timeout
51922024-10-17T22:42:49.395ZINFOtest_metrics (BackgroundTasks): background instance updater explicitly disabled background_task = instance_updater
51932024-10-17T22:42:49.395ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = instance_updater elapsed = 59.541\u{b5}s iteration = 3 status = {"destroyed_active_vmms":0,"disabled":true,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
51942024-10-17T22:42:49.396ZDEBGtest_metrics (BackgroundTasks): activating background_task = vpc_route_manager iteration = 3 reason = Timeout
51952024-10-17T22:42:49.396ZINFOtest_metrics (BackgroundTasks): VPC route manager running background_task = vpc_route_manager
51962024-10-17T22:42:49.396ZTRACtest_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet
51972024-10-17T22:42:49.396ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = vpc_route_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
51982024-10-17T22:42:49.396ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
51992024-10-17T22:42:49.397ZDEBGtest_metrics (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
52002024-10-17T22:42:49.397ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
52012024-10-17T22:42:49.398ZDEBGtest_metrics (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[::1]:12225/local/switch-id
52022024-10-17T22:42:49.398ZINFOtest_metrics (store): attempting generation update new_generation = 1 req_id = 4cc3bee4-ffce-4bc8-8095-10514b4d1dda
52032024-10-17T22:42:49.398ZINFOtest_metrics (store): updated generation new_generation = 1 req_id = 4cc3bee4-ffce-4bc8-8095-10514b4d1dda
52042024-10-17T22:42:49.399ZINFOtest_metrics (http): request completed kind = dns latency_us = 1927 local_addr = [::1]:41599 method = PUT remote_addr = [::1]:52946 req_id = 4cc3bee4-ffce-4bc8-8095-10514b4d1dda response_code = 204 uri = /config
52052024-10-17T22:42:49.399ZTRACtest_metrics (http): incoming request kind = dns local_addr = [::1]:36177 method = PUT remote_addr = [::1]:48052 req_id = b44e6f36-0c8f-41bd-91cd-d6194364e576 uri = /config
52062024-10-17T22:42:49.400ZDEBGtest_metrics (MgsClient): client response background_task = nat_v4_garbage_collector result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
52072024-10-17T22:42:49.400ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = nat_v4_garbage_collector reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
52082024-10-17T22:42:49.400ZINFOtest_metrics (store): attempting generation update new_generation = 2 req_id = b44e6f36-0c8f-41bd-91cd-d6194364e576
52092024-10-17T22:42:49.400ZINFOtest_metrics (store): updated generation new_generation = 2 req_id = b44e6f36-0c8f-41bd-91cd-d6194364e576
52102024-10-17T22:42:49.400ZTRACtest_metrics (dns): buffer buffer = Length: 76 (0x4c) bytes\n0000: ee 0b 01 00 00 01 00 00 00 00 00 01 0d 5f 65 78 ............._ex\n0010: 74 65 72 6e 61 6c 2d 64 6e 73 04 5f 74 63 70 0d ternal-dns._tcp.\n0020: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0030: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 21 00 ide.internal..!.\n0040: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:63452 req_id = 90b23fc8-746e-4349-872c-254a1668cf5e
52112024-10-17T22:42:49.400ZDEBGtest_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 60939,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_external-dns._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_external-dns._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 13,\n 95,\n 101,\n 120,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 45,\n 100,\n 110,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63452 req_id = 90b23fc8-746e-4349-872c-254a1668cf5e
52122024-10-17T22:42:49.400ZDEBGtest_metrics (store): query key key = _external-dns._tcp
52132024-10-17T22:42:49.401ZDEBGtest_metrics (store): query key key = 0e0c52d8-7b87-4b83-9828-05f7d187ffb1.host
52142024-10-17T22:42:49.401ZDEBGtest_metrics (dns): dns response additional_records = [Record { name_labels: Name("0e0c52d8-7b87-4b83-9828-05f7d187ffb1.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:63452 query = LowerQuery { name: LowerName(Name("_external-dns._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 36177, target: Name("0e0c52d8-7b87-4b83-9828-05f7d187ffb1.host.control-plane.oxide.internal") })) }] req_id = 90b23fc8-746e-4349-872c-254a1668cf5e
52152024-10-17T22:42:49.401ZDEBGtest_metrics (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[::1]:41599 generation = 1 result = Ok(Response { url: "http://[::1]:41599/config", status: 204, headers: {"x-request-id": "4cc3bee4-ffce-4bc8-8095-10514b4d1dda", "date": "Thu, 17 Oct 2024 22:42:49 GMT"} }) servers = DnsServersList { addresses: [[::1]:41599] }
52162024-10-17T22:42:49.401ZINFOtest_metrics (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[::1]:41599 generation = 1 servers = DnsServersList { addresses: [[::1]:41599] }
52172024-10-17T22:42:49.401ZINFOtest_metrics (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal generation = 1 servers = DnsServersList { addresses: [[::1]:41599] }
52182024-10-17T22:42:49.401ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 121.263503ms iteration = 4 status = {"generation":1,"server_results":{"[::1]:41599":{"Ok":null}}}
52192024-10-17T22:42:49.401ZINFOtest_metrics (http): request completed kind = dns latency_us = 2739 local_addr = [::1]:36177 method = PUT remote_addr = [::1]:48052 req_id = b44e6f36-0c8f-41bd-91cd-d6194364e576 response_code = 204 uri = /config
52202024-10-17T22:42:49.401ZDEBGtest_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52212024-10-17T22:42:49.406ZDEBGtest_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
52222024-10-17T22:42:49.406ZINFOtest_metrics (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-10-17 22:32:49.374566145 UTC pruned = {}
52232024-10-17T22:42:49.406ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 32.270805ms iteration = 2 status = {"expiration":"2024-10-17T22:32:49.374566145Z","pruned":[]}
52242024-10-17T22:42:49.407ZDEBGtest_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 36177, target: Name("0e0c52d8-7b87-4b83-9828-05f7d187ffb1.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("0e0c52d8-7b87-4b83-9828-05f7d187ffb1.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2141, tv_nsec: 176141954 } })
52252024-10-17T22:42:49.407ZTRACtest_metrics (DnsResolver): lookup_service_targets: looking up SRV target
52262024-10-17T22:42:49.408ZDEBGtest_metrics (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[::1]:36177 generation = 2 result = Ok(Response { url: "http://[::1]:36177/config", status: 204, headers: {"x-request-id": "b44e6f36-0c8f-41bd-91cd-d6194364e576", "date": "Thu, 17 Oct 2024 22:42:49 GMT"} }) servers = DnsServersList { addresses: [[::1]:36177] }
52272024-10-17T22:42:49.408ZINFOtest_metrics (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[::1]:36177 generation = 2 servers = DnsServersList { addresses: [[::1]:36177] }
52282024-10-17T22:42:49.409ZINFOtest_metrics (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external generation = 2 servers = DnsServersList { addresses: [[::1]:36177] }
52292024-10-17T22:42:49.409ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 91.287083ms iteration = 4 status = {"generation":2,"server_results":{"[::1]:36177":{"Ok":null}}}
52302024-10-17T22:42:49.410ZDEBGtest_metrics (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
52312024-10-17T22:42:49.410ZDEBGtest_metrics (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
52322024-10-17T22:42:49.410ZDEBGtest_metrics (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
52332024-10-17T22:42:49.411ZTRACtest_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig
52342024-10-17T22:42:49.411ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_internal dns_group = internal resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
52352024-10-17T22:42:49.411ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
52362024-10-17T22:42:49.411ZDEBGtest_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
52372024-10-17T22:42:49.413ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
52382024-10-17T22:42:49.415ZTRACtest_metrics (dns): buffer buffer = Length: 99 (0x63) bytes\n0000: 07 70 01 00 00 01 00 00 00 00 00 01 24 30 65 30 .p..........$0e0\n0010: 63 35 32 64 38 2d 37 62 38 37 2d 34 62 38 33 2d c52d8-7b87-4b83-\n0020: 39 38 32 38 2d 30 35 66 37 64 31 38 37 66 66 62 9828-05f7d187ffb\n0030: 31 04 68 6f 73 74 0d 63 6f 6e 74 72 6f 6c 2d 70 1.host.control-p\n0040: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0050: 6e 61 6c 00 00 1c 00 01 00 00 29 04 d0 00 00 00 nal.......).....\n0060: 00 00 00 ... kind = dns peer_addr = [::1]:59746 req_id = fdba7df5-41a3-4694-8f78-a018ef442ae9
52392024-10-17T22:42:49.415ZDEBGtest_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 1904,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("0e0c52d8-7b87-4b83-9828-05f7d187ffb1.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("0e0c52d8-7b87-4b83-9828-05f7d187ffb1.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 36,\n 48,\n 101,\n 48,\n 99,\n 53,\n 50,\n 100,\n 56,\n 45,\n 55,\n 98,\n 56,\n 55,\n 45,\n 52,\n 98,\n 56,\n 51,\n 45,\n 57,\n 56,\n 50,\n 56,\n 45,\n 48,\n 53,\n 102,\n 55,\n 100,\n 49,\n 56,\n 55,\n 102,\n 102,\n 98,\n 49,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:59746 req_id = fdba7df5-41a3-4694-8f78-a018ef442ae9
52402024-10-17T22:42:49.416ZDEBGtest_metrics (store): query key key = 0e0c52d8-7b87-4b83-9828-05f7d187ffb1.host
52412024-10-17T22:42:49.416ZDEBGtest_metrics (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:59746 query = LowerQuery { name: LowerName(Name("0e0c52d8-7b87-4b83-9828-05f7d187ffb1.host.control-plane.oxide.internal.")), original: Query { name: Name("0e0c52d8-7b87-4b83-9828-05f7d187ffb1.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("0e0c52d8-7b87-4b83-9828-05f7d187ffb1.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = fdba7df5-41a3-4694-8f78-a018ef442ae9
52422024-10-17T22:42:49.416ZDEBGtest_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52432024-10-17T22:42:49.426ZDEBGtest_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
52442024-10-17T22:42:49.427ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
52452024-10-17T22:42:49.427ZDEBGtest_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
52462024-10-17T22:42:49.427ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
52472024-10-17T22:42:49.428ZDEBGtest_metrics (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[::1]:36177] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[::1]:36177] } dns_group = external
52482024-10-17T22:42:49.428ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 54.526511ms iteration = 3 status = {"addresses":["[::1]:36177"]}
52492024-10-17T22:42:49.429ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 45.526134ms iteration = 3 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
52502024-10-17T22:42:49.435ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52512024-10-17T22:42:49.452ZDEBGtest_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
52522024-10-17T22:42:49.453ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database
52532024-10-17T22:42:49.453ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
52542024-10-17T22:42:49.476ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
52552024-10-17T22:42:49.479ZDEBGtest_metrics (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[::1]:12225/local/switch-id
52562024-10-17T22:42:49.484ZDEBGtest_metrics (MgsClient): client response background_task = bfd_manager result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
52572024-10-17T22:42:49.492ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = bfd_manager reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
52582024-10-17T22:42:49.494ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
52592024-10-17T22:42:49.494ZDEBGtest_metrics (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
52602024-10-17T22:42:49.503ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
52612024-10-17T22:42:49.503ZDEBGtest_metrics (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52622024-10-17T22:42:49.505ZDEBGtest_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
52632024-10-17T22:42:49.505ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database
52642024-10-17T22:42:49.505ZDEBGtest_metrics (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
52652024-10-17T22:42:49.505ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
52662024-10-17T22:42:49.507ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 119.820979ms iteration = 2 status = {"errors":[],"found_port_ok":[]}
52672024-10-17T22:42:49.507ZDEBGtest_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52682024-10-17T22:42:49.518ZDEBGtest_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
52692024-10-17T22:42:49.518ZDEBGtest_metrics (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
52702024-10-17T22:42:49.518ZINFOtest_metrics (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
52712024-10-17T22:42:49.519ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 132.244418ms iteration = 2 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
52722024-10-17T22:42:49.519ZDEBGtest_metrics (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52732024-10-17T22:42:49.525ZDEBGtest_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
52742024-10-17T22:42:49.526ZTRACtest_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet
52752024-10-17T22:42:49.526ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = vpc_route_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
52762024-10-17T22:42:49.526ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
52772024-10-17T22:42:49.526ZDEBGtest_metrics (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
52782024-10-17T22:42:49.526ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
52792024-10-17T22:42:49.528ZDEBGtest_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52802024-10-17T22:42:49.535ZDEBGtest_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
52812024-10-17T22:42:49.535ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
52822024-10-17T22:42:49.535ZDEBGtest_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
52832024-10-17T22:42:49.536ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
52842024-10-17T22:42:49.536ZDEBGtest_metrics (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
52852024-10-17T22:42:49.536ZDEBGtest_metrics (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
52862024-10-17T22:42:49.536ZDEBGtest_metrics (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
52872024-10-17T22:42:49.537ZTRACtest_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig
52882024-10-17T22:42:49.537ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_external dns_group = external resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
52892024-10-17T22:42:49.538ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
52902024-10-17T22:42:49.538ZDEBGtest_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
52912024-10-17T22:42:49.538ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
52922024-10-17T22:42:49.539ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database
52932024-10-17T22:42:49.543ZDEBGtest_metrics (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
52942024-10-17T22:42:49.543ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
52952024-10-17T22:42:49.543ZTRACtest_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig
52962024-10-17T22:42:49.543ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = external_endpoints resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
52972024-10-17T22:42:49.543ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
52982024-10-17T22:42:49.543ZDEBGtest_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
52992024-10-17T22:42:49.543ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
53002024-10-17T22:42:49.543ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database
53012024-10-17T22:42:49.543ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
53022024-10-17T22:42:49.543ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
53032024-10-17T22:42:49.543ZDEBGtest_metrics (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53042024-10-17T22:42:49.550ZDEBGtest_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
53052024-10-17T22:42:49.550ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
53062024-10-17T22:42:49.550ZDEBGtest_metrics (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
53072024-10-17T22:42:49.551ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
53082024-10-17T22:42:49.552ZINFOtest_metrics (BackgroundTasks): all instance checks complete background_task = instance_watcher pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
53092024-10-17T22:42:49.553ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 166.627294ms iteration = 3 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
53102024-10-17T22:42:49.575ZDEBGtest_metrics (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
53112024-10-17T22:42:49.576ZDEBGtest_metrics (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 2907eb7b-cfa4-44c5-a6f6-96e1e2e4fa6f dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
53122024-10-17T22:42:49.576ZTRACtest_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig
53132024-10-17T22:42:49.577ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_internal dns_group = internal resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
53142024-10-17T22:42:49.577ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
53152024-10-17T22:42:49.577ZDEBGtest_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
53162024-10-17T22:42:49.579ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
53172024-10-17T22:42:49.581ZTRACtest_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet
53182024-10-17T22:42:49.582ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = v2p_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
53192024-10-17T22:42:49.582ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
53202024-10-17T22:42:49.582ZDEBGtest_metrics (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
53212024-10-17T22:42:49.582ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
53222024-10-17T22:42:49.584ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database
53232024-10-17T22:42:49.584ZDEBGtest_metrics (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
53242024-10-17T22:42:49.589ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
53252024-10-17T22:42:49.590ZDEBGtest_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53262024-10-17T22:42:49.597ZDEBGtest_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
53272024-10-17T22:42:49.598ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
53282024-10-17T22:42:49.598ZDEBGtest_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
53292024-10-17T22:42:49.598ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
53302024-10-17T22:42:49.607ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 221.927753ms iteration = 2 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
53312024-10-17T22:42:49.617ZDEBGtest_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53322024-10-17T22:42:49.617ZDEBGtest_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
53332024-10-17T22:42:49.617ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
53342024-10-17T22:42:49.617ZDEBGtest_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
53352024-10-17T22:42:49.617ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
53362024-10-17T22:42:49.685ZINFOtest_metrics (BackgroundTasks): VPC route manager sled 039be560-54cc-49e3-88df-1a29dadbf913 background_task = vpc_route_manager
53372024-10-17T22:42:49.685ZDEBGtest_metrics (BackgroundTasks): client request SledAgent = 039be560-54cc-49e3-88df-1a29dadbf913 background_task = vpc_route_manager body = None method = GET uri = http://[::1]:35099/vpc-routes
53382024-10-17T22:42:49.690ZDEBGtest_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53392024-10-17T22:42:49.698ZDEBGtest_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
53402024-10-17T22:42:49.698ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
53412024-10-17T22:42:49.698ZDEBGtest_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
53422024-10-17T22:42:49.698ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
53432024-10-17T22:42:49.700ZDEBGtest_metrics (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53442024-10-17T22:42:49.705ZDEBGtest_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
53452024-10-17T22:42:49.711ZTRACtest_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet
53462024-10-17T22:42:49.711ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = v2p_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
53472024-10-17T22:42:49.712ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
53482024-10-17T22:42:49.712ZDEBGtest_metrics (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
53492024-10-17T22:42:49.712ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
53502024-10-17T22:42:49.712ZDEBGtest_metrics (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
53512024-10-17T22:42:49.712ZDEBGtest_metrics (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = f7bca40a-4301-43a3-b7a5-fe7b01c00551 dns_zone_name = oxide-dev.test found_so_far = 0 version = 2
53522024-10-17T22:42:49.712ZTRACtest_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig
53532024-10-17T22:42:49.712ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_external dns_group = external resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
53542024-10-17T22:42:49.712ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
53552024-10-17T22:42:49.713ZDEBGtest_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
53562024-10-17T22:42:49.713ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
53572024-10-17T22:42:49.713ZTRACtest_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet
53582024-10-17T22:42:49.713ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = external_endpoints resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
53592024-10-17T22:42:49.713ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
53602024-10-17T22:42:49.713ZDEBGtest_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
53612024-10-17T22:42:49.714ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
53622024-10-17T22:42:49.714ZDEBGtest_metrics (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 2907eb7b-cfa4-44c5-a6f6-96e1e2e4fa6f dns_zone_name = control-plane.oxide.internal found_so_far = 18 version = 1
53632024-10-17T22:42:49.714ZDEBGtest_metrics (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
53642024-10-17T22:42:49.714ZDEBGtest_metrics (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-10-17 22:41:52.263034 UTC dns_group = internal generation = 1
53652024-10-17T22:42:49.714ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 443.156489ms iteration = 3 status = {"generation":1}
53662024-10-17T22:42:49.714ZDEBGtest_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53672024-10-17T22:42:49.727ZDEBGtest_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
53682024-10-17T22:42:49.727ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
53692024-10-17T22:42:49.727ZDEBGtest_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
53702024-10-17T22:42:49.728ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
53712024-10-17T22:42:49.738ZDEBGtest_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53722024-10-17T22:42:49.743ZDEBGtest_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
53732024-10-17T22:42:49.743ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
53742024-10-17T22:42:49.743ZDEBGtest_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
53752024-10-17T22:42:49.744ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
53762024-10-17T22:42:49.745ZINFOtest_metrics (dropshot): accepted connection local_addr = [::1]:35099 remote_addr = [::1]:59854 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
53772024-10-17T22:42:49.745ZDEBGtest_metrics (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc uri = http://[::1]:12225/local/switch-id
53782024-10-17T22:42:49.756ZTRACtest_metrics (dropshot): incoming request local_addr = [::1]:35099 method = GET remote_addr = [::1]:59854 req_id = 6f57e49f-d875-4976-89a4-b898292aeb7c sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /vpc-routes
53792024-10-17T22:42:49.756ZDEBGtest_metrics (MgsClient): client response background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
53802024-10-17T22:42:49.756ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
53812024-10-17T22:42:49.756ZINFOtest_metrics (dropshot): request completed latency_us = 367 local_addr = [::1]:35099 method = GET remote_addr = [::1]:59854 req_id = 6f57e49f-d875-4976-89a4-b898292aeb7c response_code = 200 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /vpc-routes
53822024-10-17T22:42:49.756ZDEBGtest_metrics (BackgroundTasks): client response SledAgent = 039be560-54cc-49e3-88df-1a29dadbf913 background_task = vpc_route_manager result = Ok(Response { url: "http://[::1]:35099/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "6f57e49f-d875-4976-89a4-b898292aeb7c", "content-length": "2", "date": "Thu, 17 Oct 2024 22:42:49 GMT"} })
53832024-10-17T22:42:49.756ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
53842024-10-17T22:42:49.757ZDEBGtest_metrics (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
53852024-10-17T22:42:49.757ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
53862024-10-17T22:42:49.760ZDEBGtest_metrics (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = f7bca40a-4301-43a3-b7a5-fe7b01c00551 dns_zone_name = oxide-dev.test found_so_far = 1 version = 2
53872024-10-17T22:42:49.760ZDEBGtest_metrics (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
53882024-10-17T22:42:49.760ZDEBGtest_metrics (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-10-17 22:41:53.391320 UTC dns_group = external generation = 2
53892024-10-17T22:42:49.760ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 443.179569ms iteration = 3 status = {"generation":2}
53902024-10-17T22:42:49.780ZDEBGtest_metrics (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53912024-10-17T22:42:49.781ZDEBGtest_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
53922024-10-17T22:42:49.781ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
53932024-10-17T22:42:49.781ZDEBGtest_metrics (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
53942024-10-17T22:42:49.781ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
53952024-10-17T22:42:49.795ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = region_replacement elapsed = 409.982891ms iteration = 2 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
53962024-10-17T22:42:49.905ZDEBGtest_metrics (BackgroundTasks): client request SledAgent = 039be560-54cc-49e3-88df-1a29dadbf913 background_task = v2p_manager body = None method = GET uri = http://[::1]:35099/v2p
53972024-10-17T22:42:49.906ZDEBGtest_metrics (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide-dev.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-10-17T22:41:47.111093Z, time_modified: 2024-10-17T22:41:47.111093Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "test-suite-silo.sys.oxide-dev.test": ExternalEndpoint { silo_id: 29732a9a-d8c5-46bb-8799-9f9175aa6d42, db_silo: Silo { identity: SiloIdentity { id: 29732a9a-d8c5-46bb-8799-9f9175aa6d42, name: Name(Name("test-suite-silo")), description: "built-in recovery Silo", time_created: 2024-10-17T22:41:53.306448Z, time_modified: 2024-10-17T22:41:53.306448Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(1)) }, tls_certs: [] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide-dev.test" }, NoSiloCerts { silo_id: 29732a9a-d8c5-46bb-8799-9f9175aa6d42, dns_name: "test-suite-silo.sys.oxide-dev.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 29732a9a-d8c5-46bb-8799-9f9175aa6d42, db_silo: Silo { identity: SiloIdentity { id: 29732a9a-d8c5-46bb-8799-9f9175aa6d42, name: Name(Name("test-suite-silo")), description: "built-in recovery Silo", time_created: 2024-10-17T22:41:53.306448Z, time_modified: 2024-10-17T22:41:53.306448Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(1)) }, tls_certs: [] }) }
53982024-10-17T22:42:49.906ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 523.93748ms iteration = 3 status = {"by_dns_name":{"default-silo.sys.oxide-dev.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"test-suite-silo.sys.oxide-dev.test":{"silo_id":"29732a9a-d8c5-46bb-8799-9f9175aa6d42","tls_certs":[]}},"default_endpoint":{"silo_id":"29732a9a-d8c5-46bb-8799-9f9175aa6d42","tls_certs":[]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide-dev.test\\" has no usable certificates","silo 29732a9a-d8c5-46bb-8799-9f9175aa6d42 with DNS name \\"test-suite-silo.sys.oxide-dev.test\\" has no usable certificates"]}
53992024-10-17T22:42:49.906ZINFOtest_metrics (dropshot): accepted connection local_addr = [::1]:35099 remote_addr = [::1]:59856 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
54002024-10-17T22:42:49.907ZTRACtest_metrics (dropshot): incoming request local_addr = [::1]:35099 method = GET remote_addr = [::1]:59856 req_id = d0e45505-d8ec-43a1-be3f-f2cc6986e6a8 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /v2p
54012024-10-17T22:42:49.908ZINFOtest_metrics (dropshot): request completed latency_us = 612 local_addr = [::1]:35099 method = GET remote_addr = [::1]:59856 req_id = d0e45505-d8ec-43a1-be3f-f2cc6986e6a8 response_code = 200 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /v2p
54022024-10-17T22:42:49.909ZDEBGtest_metrics (BackgroundTasks): client response SledAgent = 039be560-54cc-49e3-88df-1a29dadbf913 background_task = v2p_manager result = Ok(Response { url: "http://[::1]:35099/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "d0e45505-d8ec-43a1-be3f-f2cc6986e6a8", "content-length": "2", "date": "Thu, 17 Oct 2024 22:42:49 GMT"} })
54032024-10-17T22:42:49.909ZINFOtest_metrics (BackgroundTasks): found opte v2p mappings background_task = v2p_manager interfaces = {} sled = sim-039be560
54042024-10-17T22:42:49.909ZINFOtest_metrics (BackgroundTasks): v2p mappings to delete background_task = v2p_manager mappings = [] sled = sim-039be560
54052024-10-17T22:42:49.909ZINFOtest_metrics (BackgroundTasks): v2p mappings to add background_task = v2p_manager mappings = [] sled = sim-039be560
54062024-10-17T22:42:49.909ZDEBGtest_metrics (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = v2p_manager body = None method = GET uri = http://[::1]:40953/v2p
54072024-10-17T22:42:49.910ZINFOtest_metrics (dropshot): accepted connection local_addr = [::1]:40953 remote_addr = [::1]:57554 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
54082024-10-17T22:42:49.910ZTRACtest_metrics (dropshot): incoming request local_addr = [::1]:40953 method = GET remote_addr = [::1]:57554 req_id = 34f07619-2349-46f7-ae31-9875d5fb9e77 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /v2p
54092024-10-17T22:42:49.911ZINFOtest_metrics (dropshot): request completed latency_us = 238 local_addr = [::1]:40953 method = GET remote_addr = [::1]:57554 req_id = 34f07619-2349-46f7-ae31-9875d5fb9e77 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /v2p
54102024-10-17T22:42:49.911ZDEBGtest_metrics (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = v2p_manager result = Ok(Response { url: "http://[::1]:40953/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "34f07619-2349-46f7-ae31-9875d5fb9e77", "content-length": "2", "date": "Thu, 17 Oct 2024 22:42:49 GMT"} })
54112024-10-17T22:42:49.911ZINFOtest_metrics (BackgroundTasks): found opte v2p mappings background_task = v2p_manager interfaces = {} sled = sim-b6d65341
54122024-10-17T22:42:49.911ZINFOtest_metrics (BackgroundTasks): v2p mappings to delete background_task = v2p_manager mappings = [] sled = sim-b6d65341
54132024-10-17T22:42:49.911ZINFOtest_metrics (BackgroundTasks): v2p mappings to add background_task = v2p_manager mappings = [] sled = sim-b6d65341
54142024-10-17T22:42:49.913ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 529.499904ms iteration = 3 status = {}
54152024-10-17T22:42:49.935ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database
54162024-10-17T22:42:49.935ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
54172024-10-17T22:42:49.936ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
54182024-10-17T22:42:49.939ZTRACtest_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 039be560-54cc-49e3-88df-1a29dadbf913, lookup_type: ById(039be560-54cc-49e3-88df-1a29dadbf913) }
54192024-10-17T22:42:49.939ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_zone_nat_tracker resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
54202024-10-17T22:42:49.939ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database
54212024-10-17T22:42:49.939ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
54222024-10-17T22:42:49.939ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
54232024-10-17T22:42:49.941ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54242024-10-17T22:42:49.947ZDEBGtest_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 039be560-54cc-49e3-88df-1a29dadbf913, lookup_type: ById(039be560-54cc-49e3-88df-1a29dadbf913) } result = Ok(())
54252024-10-17T22:42:49.949ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database
54262024-10-17T22:42:49.949ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
54272024-10-17T22:42:49.949ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
54282024-10-17T22:42:49.957ZTRACtest_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b6d65341-167c-41df-9b5c-41cded99c229, lookup_type: ById(b6d65341-167c-41df-9b5c-41cded99c229) }
54292024-10-17T22:42:49.957ZTRACtest_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_zone_nat_tracker resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
54302024-10-17T22:42:49.957ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database
54312024-10-17T22:42:49.957ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
54322024-10-17T22:42:49.958ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
54332024-10-17T22:42:49.966ZINFOtest_metrics (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager sled = sim-039be560
54342024-10-17T22:42:49.966ZDEBGtest_metrics (BackgroundTasks): client request SledAgent = 039be560-54cc-49e3-88df-1a29dadbf913 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[::1]:35099/eip-gateways
54352024-10-17T22:42:49.967ZTRACtest_metrics (dropshot): incoming request local_addr = [::1]:35099 method = PUT remote_addr = [::1]:59854 req_id = e4c3a6ad-2558-4c6f-a3e7-d64e4ae6c627 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /eip-gateways
54362024-10-17T22:42:49.967ZINFOtest_metrics (dropshot): request completed latency_us = 271 local_addr = [::1]:35099 method = PUT remote_addr = [::1]:59854 req_id = e4c3a6ad-2558-4c6f-a3e7-d64e4ae6c627 response_code = 204 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /eip-gateways
54372024-10-17T22:42:49.968ZDEBGtest_metrics (BackgroundTasks): client response SledAgent = 039be560-54cc-49e3-88df-1a29dadbf913 background_task = vpc_route_manager result = Ok(Response { url: "http://[::1]:35099/eip-gateways", status: 204, headers: {"x-request-id": "e4c3a6ad-2558-4c6f-a3e7-d64e4ae6c627", "date": "Thu, 17 Oct 2024 22:42:49 GMT"} })
54382024-10-17T22:42:49.970ZINFOtest_metrics (BackgroundTasks): VPC route manager sled b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager
54392024-10-17T22:42:49.970ZDEBGtest_metrics (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager body = None method = GET uri = http://[::1]:40953/vpc-routes
54402024-10-17T22:42:49.971ZDEBGtest_metrics (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54412024-10-17T22:42:49.976ZDEBGtest_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b6d65341-167c-41df-9b5c-41cded99c229, lookup_type: ById(b6d65341-167c-41df-9b5c-41cded99c229) } result = Ok(())
54422024-10-17T22:42:49.976ZERROtest_metrics (BackgroundTasks): ipv6 addresses for service zone nat not implemented background_task = service_zone_nat_tracker
54432024-10-17T22:42:49.976ZERROtest_metrics (BackgroundTasks): generated config for fewer than the minimum allowed number of dns zones background_task = service_zone_nat_tracker
54442024-10-17T22:42:49.977ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 593.05375ms iteration = 3 status = {"error":"generated config for fewer than the minimum allowed number of dns zones"}
54452024-10-17T22:42:49.977ZINFOtest_metrics (dropshot): accepted connection local_addr = [::1]:40953 remote_addr = [::1]:57556 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
54462024-10-17T22:42:49.978ZTRACtest_metrics (dropshot): incoming request local_addr = [::1]:40953 method = GET remote_addr = [::1]:57556 req_id = fda0c038-e71c-468d-a2dc-e9f6e1da752d sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc-routes
54472024-10-17T22:42:49.983ZINFOtest_metrics (dropshot): request completed latency_us = 4836 local_addr = [::1]:40953 method = GET remote_addr = [::1]:57556 req_id = fda0c038-e71c-468d-a2dc-e9f6e1da752d response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc-routes
54482024-10-17T22:42:49.984ZDEBGtest_metrics (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager result = Ok(Response { url: "http://[::1]:40953/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "fda0c038-e71c-468d-a2dc-e9f6e1da752d", "content-length": "2", "date": "Thu, 17 Oct 2024 22:42:49 GMT"} })
54492024-10-17T22:42:49.984ZTRACtest_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
54502024-10-17T22:42:49.984ZDEBGtest_metrics (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
54512024-10-17T22:42:49.985ZDEBGtest_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
54522024-10-17T22:42:50.006ZINFOtest_metrics (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager sled = sim-b6d65341
54532024-10-17T22:42:50.007ZDEBGtest_metrics (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[::1]:40953/eip-gateways
54542024-10-17T22:42:50.007ZTRACtest_metrics (dropshot): incoming request local_addr = [::1]:40953 method = PUT remote_addr = [::1]:57556 req_id = 7293d5b8-51c9-4046-acbc-c04275e059b0 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /eip-gateways
54552024-10-17T22:42:50.007ZINFOtest_metrics (dropshot): request completed latency_us = 314 local_addr = [::1]:40953 method = PUT remote_addr = [::1]:57556 req_id = 7293d5b8-51c9-4046-acbc-c04275e059b0 response_code = 204 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /eip-gateways
54562024-10-17T22:42:50.007ZDEBGtest_metrics (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager result = Ok(Response { url: "http://[::1]:40953/eip-gateways", status: 204, headers: {"x-request-id": "7293d5b8-51c9-4046-acbc-c04275e059b0", "date": "Thu, 17 Oct 2024 22:42:49 GMT"} })
54572024-10-17T22:42:50.008ZDEBGtest_metrics (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 617.065608ms iteration = 3 status = {}
54582024-10-17T22:42:50.186ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
54592024-10-17T22:42:50.189ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
54602024-10-17T22:42:50.244ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
54612024-10-17T22:42:50.244ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
54622024-10-17T22:42:50.245ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
54632024-10-17T22:42:50.253ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
54642024-10-17T22:42:50.253ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:50.187321165","datum":52}\n
54652024-10-17T22:42:50.257ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
54662024-10-17T22:42:50.257ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
54672024-10-17T22:42:51.188ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
54682024-10-17T22:42:51.189ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
54692024-10-17T22:42:51.246ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
54702024-10-17T22:42:51.246ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
54712024-10-17T22:42:51.246ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
54722024-10-17T22:42:51.252ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
54732024-10-17T22:42:51.253ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:51.187605941","datum":53}\n
54742024-10-17T22:42:51.258ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
54752024-10-17T22:42:51.258ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
54762024-10-17T22:42:51.395ZDEBGtest_metrics (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[::1]:12225/local/switch-id
54772024-10-17T22:42:51.395ZDEBGtest_metrics (MgsClient): client response background_task = nat_v4_garbage_collector result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
54782024-10-17T22:42:51.395ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = nat_v4_garbage_collector reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
54792024-10-17T22:42:51.484ZDEBGtest_metrics (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[::1]:12225/local/switch-id
54802024-10-17T22:42:51.486ZDEBGtest_metrics (MgsClient): client response background_task = bfd_manager result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
54812024-10-17T22:42:51.486ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = bfd_manager reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
54822024-10-17T22:42:51.757ZDEBGtest_metrics (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc uri = http://[::1]:12225/local/switch-id
54832024-10-17T22:42:51.758ZDEBGtest_metrics (MgsClient): client response background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
54842024-10-17T22:42:51.758ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
54852024-10-17T22:42:52.187ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
54862024-10-17T22:42:52.190ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
54872024-10-17T22:42:52.243ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
54882024-10-17T22:42:52.243ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
54892024-10-17T22:42:52.245ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
54902024-10-17T22:42:52.249ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
54912024-10-17T22:42:52.252ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:52.187788474","datum":54}\n
54922024-10-17T22:42:52.257ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
54932024-10-17T22:42:52.258ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
54942024-10-17T22:42:53.186ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
54952024-10-17T22:42:53.189ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
54962024-10-17T22:42:53.244ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
54972024-10-17T22:42:53.244ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
54982024-10-17T22:42:53.245ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
54992024-10-17T22:42:53.252ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
55002024-10-17T22:42:53.252ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:53.187207639","datum":55}\n
55012024-10-17T22:42:53.259ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
55022024-10-17T22:42:53.259ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
55032024-10-17T22:42:53.397ZDEBGtest_metrics (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[::1]:12225/local/switch-id
55042024-10-17T22:42:53.399ZDEBGtest_metrics (MgsClient): client response background_task = nat_v4_garbage_collector result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
55052024-10-17T22:42:53.399ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = nat_v4_garbage_collector reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
55062024-10-17T22:42:53.487ZDEBGtest_metrics (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[::1]:12225/local/switch-id
55072024-10-17T22:42:53.487ZDEBGtest_metrics (MgsClient): client response background_task = bfd_manager result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
55082024-10-17T22:42:53.487ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = bfd_manager reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
55092024-10-17T22:42:53.758ZDEBGtest_metrics (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc uri = http://[::1]:12225/local/switch-id
55102024-10-17T22:42:53.759ZDEBGtest_metrics (MgsClient): client response background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
55112024-10-17T22:42:53.759ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
55122024-10-17T22:42:54.186ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
55132024-10-17T22:42:54.189ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
55142024-10-17T22:42:54.244ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
55152024-10-17T22:42:54.244ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
55162024-10-17T22:42:54.245ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
55172024-10-17T22:42:54.251ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
55182024-10-17T22:42:54.251ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:54.187136636","datum":56}\n
55192024-10-17T22:42:54.257ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
55202024-10-17T22:42:54.258ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
55212024-10-17T22:42:55.186ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
55222024-10-17T22:42:55.189ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
55232024-10-17T22:42:55.243ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
55242024-10-17T22:42:55.243ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
55252024-10-17T22:42:55.244ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
55262024-10-17T22:42:55.249ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
55272024-10-17T22:42:55.250ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:55.186926119","datum":57}\n
55282024-10-17T22:42:55.256ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
55292024-10-17T22:42:55.256ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
55302024-10-17T22:42:55.399ZDEBGtest_metrics (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[::1]:12225/local/switch-id
55312024-10-17T22:42:55.399ZDEBGtest_metrics (MgsClient): client response background_task = nat_v4_garbage_collector result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
55322024-10-17T22:42:55.399ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = nat_v4_garbage_collector reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
55332024-10-17T22:42:55.490ZDEBGtest_metrics (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[::1]:12225/local/switch-id
55342024-10-17T22:42:55.490ZDEBGtest_metrics (MgsClient): client response background_task = bfd_manager result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
55352024-10-17T22:42:55.490ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = bfd_manager reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
55362024-10-17T22:42:55.761ZDEBGtest_metrics (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc uri = http://[::1]:12225/local/switch-id
55372024-10-17T22:42:55.761ZDEBGtest_metrics (MgsClient): client response background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
55382024-10-17T22:42:55.762ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
55392024-10-17T22:42:56.186ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
55402024-10-17T22:42:56.189ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
55412024-10-17T22:42:56.247ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
55422024-10-17T22:42:56.247ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
55432024-10-17T22:42:56.247ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
55442024-10-17T22:42:56.249ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
55452024-10-17T22:42:56.250ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:56.187398669","datum":58}\n
55462024-10-17T22:42:56.256ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
55472024-10-17T22:42:56.256ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
55482024-10-17T22:42:56.465ZTRACtest_metrics (dns): buffer buffer = Length: 79 (0x4f) bytes\n0000: ef ca 01 00 00 01 00 00 00 00 00 01 10 5f 63 72 ............._cr\n0010: 75 63 69 62 6c 65 2d 70 61 6e 74 72 79 04 5f 74 ucible-pantry._t\n0020: 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 cp.control-plane\n0030: 05 6f 78 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 .oxide.internal.\n0040: 00 21 00 01 00 00 29 04 d0 00 00 00 00 00 00 .!....)........ kind = dns peer_addr = [::1]:51495 req_id = cfbb8ee5-bdc4-4cb7-b56e-94d262aec7a9
55492024-10-17T22:42:56.466ZDEBGtest_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 61386,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_crucible-pantry._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_crucible-pantry._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 16,\n 95,\n 99,\n 114,\n 117,\n 99,\n 105,\n 98,\n 108,\n 101,\n 45,\n 112,\n 97,\n 110,\n 116,\n 114,\n 121,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51495 req_id = cfbb8ee5-bdc4-4cb7-b56e-94d262aec7a9
55502024-10-17T22:42:56.466ZDEBGtest_metrics (store): query key key = _crucible-pantry._tcp
55512024-10-17T22:42:56.467ZDEBGtest_metrics (store): query key key = 4cc306e2-a646-4794-a23e-b059c98afee3.host
55522024-10-17T22:42:56.467ZDEBGtest_metrics (dns): dns response additional_records = [Record { name_labels: Name("4cc306e2-a646-4794-a23e-b059c98afee3.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:51495 query = LowerQuery { name: LowerName(Name("_crucible-pantry._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_crucible-pantry._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_crucible-pantry._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 38587, target: Name("4cc306e2-a646-4794-a23e-b059c98afee3.host.control-plane.oxide.internal") })) }] req_id = cfbb8ee5-bdc4-4cb7-b56e-94d262aec7a9
55532024-10-17T22:42:56.467ZTRACtest_metrics (dns): buffer buffer = Length: 99 (0x63) bytes\n0000: 4f 38 01 00 00 01 00 00 00 00 00 01 24 34 63 63 O8..........$4cc\n0010: 33 30 36 65 32 2d 61 36 34 36 2d 34 37 39 34 2d 306e2-a646-4794-\n0020: 61 32 33 65 2d 62 30 35 39 63 39 38 61 66 65 65 a23e-b059c98afee\n0030: 33 04 68 6f 73 74 0d 63 6f 6e 74 72 6f 6c 2d 70 3.host.control-p\n0040: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0050: 6e 61 6c 00 00 1c 00 01 00 00 29 04 d0 00 00 00 nal.......).....\n0060: 00 00 00 ... kind = dns peer_addr = [::1]:64557 req_id = 9a62f836-f0e0-491b-b00c-3cad0d5b70ba
55542024-10-17T22:42:56.467ZDEBGtest_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 20280,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("4cc306e2-a646-4794-a23e-b059c98afee3.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("4cc306e2-a646-4794-a23e-b059c98afee3.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 36,\n 52,\n 99,\n 99,\n 51,\n 48,\n 54,\n 101,\n 50,\n 45,\n 97,\n 54,\n 52,\n 54,\n 45,\n 52,\n 55,\n 57,\n 52,\n 45,\n 97,\n 50,\n 51,\n 101,\n 45,\n 98,\n 48,\n 53,\n 57,\n 99,\n 57,\n 56,\n 97,\n 102,\n 101,\n 101,\n 51,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64557 req_id = 9a62f836-f0e0-491b-b00c-3cad0d5b70ba
55552024-10-17T22:42:56.467ZDEBGtest_metrics (store): query key key = 4cc306e2-a646-4794-a23e-b059c98afee3.host
55562024-10-17T22:42:56.467ZDEBGtest_metrics (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:64557 query = LowerQuery { name: LowerName(Name("4cc306e2-a646-4794-a23e-b059c98afee3.host.control-plane.oxide.internal.")), original: Query { name: Name("4cc306e2-a646-4794-a23e-b059c98afee3.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("4cc306e2-a646-4794-a23e-b059c98afee3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 9a62f836-f0e0-491b-b00c-3cad0d5b70ba
55572024-10-17T22:42:56.665ZTRACtest_metrics (dropshot): incoming request kind = pantry local_addr = [::1]:38587 method = GET remote_addr = [::1]:42120 req_id = bcdf8740-478c-4fe0-8279-b1ad75f59070 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
55582024-10-17T22:42:56.666ZINFOtest_metrics (dropshot): request completed kind = pantry latency_us = 233 local_addr = [::1]:38587 method = GET remote_addr = [::1]:42120 req_id = bcdf8740-478c-4fe0-8279-b1ad75f59070 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
55592024-10-17T22:42:56.697ZTRACtest_metrics (dropshot): incoming request kind = pantry local_addr = [::1]:38587 method = GET remote_addr = [::1]:42122 req_id = 0279f600-79bf-44ef-9475-671659730f7a sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
55602024-10-17T22:42:56.697ZINFOtest_metrics (dropshot): request completed kind = pantry latency_us = 252 local_addr = [::1]:38587 method = GET remote_addr = [::1]:42122 req_id = 0279f600-79bf-44ef-9475-671659730f7a response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
55612024-10-17T22:42:56.747ZTRACtest_metrics (dropshot): incoming request kind = pantry local_addr = [::1]:38587 method = GET remote_addr = [::1]:42124 req_id = 4ded67e5-57b3-47e6-9e22-7ba6787d6c2f sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
55622024-10-17T22:42:56.747ZINFOtest_metrics (dropshot): request completed kind = pantry latency_us = 288 local_addr = [::1]:38587 method = GET remote_addr = [::1]:42124 req_id = 4ded67e5-57b3-47e6-9e22-7ba6787d6c2f response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
55632024-10-17T22:42:56.839ZTRACtest_metrics (dropshot): incoming request kind = pantry local_addr = [::1]:38587 method = GET remote_addr = [::1]:42126 req_id = ddd5c92b-7f4f-40dc-9386-a12de8c171e1 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
55642024-10-17T22:42:56.839ZTRACtest_metrics (dropshot): incoming request kind = pantry local_addr = [::1]:38587 method = GET remote_addr = [::1]:42128 req_id = 742c119a-e122-4224-9a6d-16dfa1fc8d9d sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
55652024-10-17T22:42:56.839ZINFOtest_metrics (dropshot): request completed kind = pantry latency_us = 379 local_addr = [::1]:38587 method = GET remote_addr = [::1]:42126 req_id = ddd5c92b-7f4f-40dc-9386-a12de8c171e1 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
55662024-10-17T22:42:56.839ZINFOtest_metrics (dropshot): request completed kind = pantry latency_us = 327 local_addr = [::1]:38587 method = GET remote_addr = [::1]:42128 req_id = 742c119a-e122-4224-9a6d-16dfa1fc8d9d response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
55672024-10-17T22:42:56.871ZTRACtest_metrics (dropshot): incoming request kind = pantry local_addr = [::1]:38587 method = GET remote_addr = [::1]:42130 req_id = 8b425390-9557-4232-988c-a0a8ea9a5e21 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
55682024-10-17T22:42:56.871ZINFOtest_metrics (dropshot): request completed kind = pantry latency_us = 266 local_addr = [::1]:38587 method = GET remote_addr = [::1]:42130 req_id = 8b425390-9557-4232-988c-a0a8ea9a5e21 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
55692024-10-17T22:42:56.891ZTRACtest_metrics (dropshot): incoming request kind = pantry local_addr = [::1]:38587 method = GET remote_addr = [::1]:42132 req_id = ab0f7b4c-558b-489a-bef2-175f8ab60e48 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
55702024-10-17T22:42:56.891ZINFOtest_metrics (dropshot): request completed kind = pantry latency_us = 254 local_addr = [::1]:38587 method = GET remote_addr = [::1]:42132 req_id = ab0f7b4c-558b-489a-bef2-175f8ab60e48 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
55712024-10-17T22:42:56.925ZTRACtest_metrics (dropshot): incoming request kind = pantry local_addr = [::1]:38587 method = GET remote_addr = [::1]:42134 req_id = 55e8df57-d9cc-454e-bfc5-ba800a627a53 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
55722024-10-17T22:42:56.925ZINFOtest_metrics (dropshot): request completed kind = pantry latency_us = 248 local_addr = [::1]:38587 method = GET remote_addr = [::1]:42134 req_id = 55e8df57-d9cc-454e-bfc5-ba800a627a53 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
55732024-10-17T22:42:57.163ZDEBGtest_metrics (collection-task): reporting oximeter self-collection statistics address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
55742024-10-17T22:42:57.187ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
55752024-10-17T22:42:57.189ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
55762024-10-17T22:42:57.244ZDEBGtest_metrics (results-sink): inserting 2 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
55772024-10-17T22:42:57.244ZDEBGtest_metrics (clickhouse-client): unrolling 2 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
55782024-10-17T22:42:57.245ZDEBGtest_metrics (clickhouse-client): retrieving timeseries schema from database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
55792024-10-17T22:42:57.245ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = SELECT * FROM oximeter.timeseries_schema WHERE timeseries_name NOT IN ('integration_target:integration_metric') FORMAT JSONEachRow;
55802024-10-17T22:42:57.250ZTRACtest_metrics (clickhouse-client): no new timeseries schema in database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
55812024-10-17T22:42:57.250ZDEBGtest_metrics (clickhouse-client): new timeseries schema collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 schema = {"timeseries_name":"oximeter_collector:collections","fields.name":["base_route","collector_id","collector_ip","collector_port","producer_id","producer_ip","producer_port"],"fields.type":["String","Uuid","IpAddr","U16","Uuid","IpAddr","U16"],"fields.source":["Metric","Target","Target","Target","Metric","Metric","Metric"],"datum_type":"CumulativeU64","created":"2024-10-17 22:42:57.244917181"} timeseries_name = oximeter_collector:collections
55822024-10-17T22:42:57.251ZDEBGtest_metrics (clickhouse-client): inserting 1 new timeseries schema collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
55832024-10-17T22:42:57.251ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.timeseries_schema FORMAT JSONEachRow\n{"timeseries_name":"oximeter_collector:collections","fields.name":["base_route","collector_id","collector_ip","collector_port","producer_id","producer_ip","producer_port"],"fields.type":["String","Uuid","IpAddr","U16","Uuid","IpAddr","U16"],"fields.source":["Metric","Target","Target","Target","Metric","Metric","Metric"],"datum_type":"CumulativeU64","created":"2024-10-17 22:42:57.244917181"}\n
55842024-10-17T22:42:57.258ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_ipaddr FORMAT JSONEachRow\n{"timeseries_name":"oximeter_collector:collections","timeseries_key":6430432085080837116,"field_name":"collector_ip","field_value":"::1"}\n{"timeseries_name":"oximeter_collector:collections","timeseries_key":6430432085080837116,"field_name":"producer_ip","field_value":"::1"}\n
55852024-10-17T22:42:57.267ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_ipaddr
55862024-10-17T22:42:57.267ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"oximeter_collector:collections","timeseries_key":6430432085080837116,"field_name":"base_route","field_value":""}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
55872024-10-17T22:42:57.274ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 3 table_name = oximeter.fields_string
55882024-10-17T22:42:57.274ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_u16 FORMAT JSONEachRow\n{"timeseries_name":"oximeter_collector:collections","timeseries_key":6430432085080837116,"field_name":"collector_port","field_value":0}\n{"timeseries_name":"oximeter_collector:collections","timeseries_key":6430432085080837116,"field_name":"producer_port","field_value":42465}\n
55892024-10-17T22:42:57.279ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_u16
55902024-10-17T22:42:57.279ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_uuid FORMAT JSONEachRow\n{"timeseries_name":"oximeter_collector:collections","timeseries_key":6430432085080837116,"field_name":"collector_id","field_value":"39e6175b-4df2-4730-b11d-cbc1e60a2e78"}\n{"timeseries_name":"oximeter_collector:collections","timeseries_key":6430432085080837116,"field_name":"producer_id","field_value":"a6458b7d-87c3-4483-be96-854d814c20de"}\n
55912024-10-17T22:42:57.286ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_uuid
55922024-10-17T22:42:57.286ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_cumulativeu64 FORMAT JSONEachRow\n{"timeseries_name":"oximeter_collector:collections","timeseries_key":6430432085080837116,"start_time":"2024-10-17 22:41:57.162920430","timestamp":"2024-10-17 22:42:57.163506734","datum":59}\n
55932024-10-17T22:42:57.290ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_cumulativeu64
55942024-10-17T22:42:57.291ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:57.187660925","datum":59}\n
55952024-10-17T22:42:57.299ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
55962024-10-17T22:42:57.299ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
55972024-10-17T22:42:57.401ZDEBGtest_metrics (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[::1]:12225/local/switch-id
55982024-10-17T22:42:57.401ZDEBGtest_metrics (MgsClient): client response background_task = nat_v4_garbage_collector result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
55992024-10-17T22:42:57.402ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = nat_v4_garbage_collector reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
56002024-10-17T22:42:57.493ZDEBGtest_metrics (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[::1]:12225/local/switch-id
56012024-10-17T22:42:57.493ZDEBGtest_metrics (MgsClient): client response background_task = bfd_manager result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
56022024-10-17T22:42:57.493ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = bfd_manager reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
56032024-10-17T22:42:57.762ZDEBGtest_metrics (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc uri = http://[::1]:12225/local/switch-id
56042024-10-17T22:42:57.763ZDEBGtest_metrics (MgsClient): client response background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
56052024-10-17T22:42:57.764ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
56062024-10-17T22:42:58.187ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
56072024-10-17T22:42:58.189ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
56082024-10-17T22:42:58.244ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
56092024-10-17T22:42:58.244ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
56102024-10-17T22:42:58.245ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
56112024-10-17T22:42:58.253ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
56122024-10-17T22:42:58.253ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:58.187827288","datum":60}\n
56132024-10-17T22:42:58.261ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
56142024-10-17T22:42:58.261ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
56152024-10-17T22:42:59.188ZDEBGtest_metrics (collection-task): collecting from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
56162024-10-17T22:42:59.191ZDEBGtest_metrics (collection-task): collected results from producer address = [::1]:42465 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
56172024-10-17T22:42:59.244ZDEBGtest_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
56182024-10-17T22:42:59.244ZDEBGtest_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8
56192024-10-17T22:42:59.245ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.fields_string FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"target_name","field_value":"integration-test-target"}\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"field_name":"metric_name","field_value":"integration-test-metric"}\n
56202024-10-17T22:42:59.249ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 2 table_name = oximeter.fields_string
56212024-10-17T22:42:59.249ZTRACtest_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 sql = INSERT INTO oximeter.measurements_i64 FORMAT JSONEachRow\n{"timeseries_name":"integration_target:integration_metric","timeseries_key":3277211643095776014,"timestamp":"2024-10-17 22:42:59.189059436","datum":61}\n
56222024-10-17T22:42:59.254ZDEBGtest_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 8fddc7a9-b5cf-4db6-bd24-2fff47ce57c8 n_rows = 1 table_name = oximeter.measurements_i64
56232024-10-17T22:42:59.254ZTRACtest_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
56242024-10-17T22:42:59.402ZDEBGtest_metrics (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[::1]:12225/local/switch-id
56252024-10-17T22:42:59.402ZDEBGtest_metrics (MgsClient): client response background_task = nat_v4_garbage_collector result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
56262024-10-17T22:42:59.403ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = nat_v4_garbage_collector reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1
56272024-10-17T22:42:59.497ZDEBGtest_metrics (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[::1]:12225/local/switch-id
56282024-10-17T22:42:59.497ZDEBGtest_metrics (MgsClient): client response background_task = bfd_manager result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
56292024-10-17T22:42:59.497ZWARNtest_metrics (BackgroundTasks): failed to identify switch slot for dendrite, will retry in 2 seconds background_task = bfd_manager reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id) zone_address = ::1